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WEAPON  SYSTEM-SPECIFIC  OCCUPATIONAL  ANALYSIS 


I.  INTRODUCTION 


This  report  is  the  third  of  a  series  considering  technologies  for  enhancing  methods 
which  identify,  describe,  and  analyze  the  work  performed  in  maintenance  specialties.  The 
research  reported  is  directed  specifically  at  increasing  the  specificity  of  data  available  from 
maintenance  Job  Inventories,  developed  and  administered  by  the  USAF  Occupational 
Measurement  Center  (USAFOMC),  that  constitute  the  occupational  survey  task  data  base 
(OSM).  There  were  two  goals  of  the  research. 

The  first  goal  was  to  test  and  recommend  enhancements  of  Semantic-Assisted  Analysis 
Technology  (SAAT)  procedures  for  mapping  existing  task  data,  from  past  occupational 
surveys  of  Air  Force  Specialties  (AFS),  and  weapon  system-specific  task  data,  from  existing 
maintenance  data  bases  (primarily  the  Maintenance  Data  Collection  System  (MDCS)).  The 
objective  of  this  effort  was  to  capitalize  on  the  large  amount  of  OSM  data  available  and  to 
associate  the  job  survey  personnel  and  training  data  more  clearly  with  specific  weapon 
system  components. 

The  second  goal  was  to  identify  and  describe  alternative  strategies  and  assess  their 
feasibility  for  developing  and  administering  future  USAF  Job  Inventories.  These  inventories 
will  include  weapons  system-specific  maintenance  tasks  and  data  based  on  MDCS  Work  Unit 
Code  (WUC)  equipment  identification  or  nomenclature  and  Action  Taken  Codes  (ATC). 

An  approach  centered  around  the  development  and  use  of  a  single  computer-generated 
Job  Inventory  from  MDCS  WUCs  and  ATCs  for  collecting  data  from  AFS  job  incumbents 
is  proposed.  Use  of  a  single  inventory  is  (1)  more  cost  effective;  (2)  essential  for  and  can 
be  employed  in  both  mail  and  computer  administration;  (3)  provides  job  incumbents 
opportunity  to  respond  to  each  system  they  maintain;  (4)  facilitates  analysis  using  the 
Comprehensive  Occupational  Data  Analysis  Programs  (CODAP);  (5)  provides  for 
comparability  of  task  factor  information  across  aircraft  for  which  an  AFS  has  responsibility; 
(6)  provides  task  data  not  specific  to  any  single  aircraft  (generic  data)  for  users  who  require 
this  kind  of  task  data;  and  (7)  provides  equipment-specific  task  data  by  specific  aircraft  for 
users  needing  more  detailed  task  data.  The  approach  is  based  on  the  use  of  two  levels  of 
task  description  to  which  job  incumbents  respond.  Responses  to  WUC-specific  tasks  would 
provide  equipment-specific  task  data  by  aircraft.  Generic  task  data  would  serve  two 
purposes:  they  are  the  key  to  aggregation  of  data  across  the  work  of  an  AFS,  and  they 
provide  a  higher  level  of  job-task  information. 
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The  proposed  approach  derives  from  careful  consideration  of  the  following  factors: 
background  and  requirements;  task  characteristics  essaitial  for  collection  of  reliable  job-task 
data;  present  occupational  survey  methodology  and  practices;  the  scope  of  the  requirement 
for  WUC-spedfic  tasks;  user  data  requirements;  and  sources  for  developing  both  specific  and 
generic  tasks.  These  factors  are  discussed  in  the  following  sections, 

n.  BACKGROUND 


This  section  describes  the  requirements  for  more  weapon  system-specific  data  and  the 
research  to  date  to  provide  for  mapping  occupational  survey  and  MDCS  data  as  well  as  a 
prototype  methodology  for  developing  occupational  survey  task  lists  from  maintenance  data. 

Requirements 

The  present  operational  program  for  Air  Force  (AF)  occupational  analysis  involves 
the  use  of  job  inventories  in  AF-wide  occupational  surveys.  Job  incumbents  identify  the 
tasks  they  perform  in  their  current  jobs,  rate  the  relative  amount  of  time  they  spend 
performing  each  of  the  tasks,  indicate  what  equipment  they  work  on,  and  also  provide  a 
variety  of  task  factor,  demographic,  and  personnel  data.  The  collective  results  of  these 
surveys  comprise  the  only  AF  data  base  in  which  personnel  information  is  linked  to  specific 
task  data,  providing  unique  capabilities  for  integrating  personnel  and  training  planning.  The 
present  products  have  significant  impacts  as  personnel  and  training  decision  aids  in 
establishing  aptitude  requirements,  determining  training  priorities,  and  confirming  the  need 
for  Air  Force  Specialty  (AFS)  shreds  or  specialized  training.  With  major  changes  and 
consolidations  being  planned  for  Air  Force  maintenance  specialties,  the  survey  data  base  is 
an  immediate  source  of  vital  information  needed  for  AFS  restructuring. 

Especially  important  is  the  need  for  a  data  base  to  aid  in  making  personnel  and 
training  decisions  on  AFSs  needed  for  new  weapon  systems.  To  serve  this  purpose  and  to 
be  compatible  with  comparability  analysis  used  in  the  design  of  a  new  system  (Mayer  & 
York,  1974),  the  job  survey  data  must  be  able  to  provide  job-task  data  specific  to  each 
weapon  system-data  uniquely  relatable  to  specific  systems  and  weapon  systems  components. 
While  this  capability  now  exists  to  a  large  extent  for  a  few  AFSs  which  relate  to  a  single 
weapon  system,  all  or  nearly  aU  job  inventories  contain  some  "generic"  task  statemaits 
covering  a  set  of  related  tasks  with  varying  losses  of  specificity.  This  situation  is  especially 
true  for  AFSs  covering  multiple  weapon  systems  because  of  the  need  to  stay  within  the 
practical  limits  of  length  for  a  mail  survey  instrument. 


The  problem  is  that  generic  task  statements  may  obscure  significant  differences  in  task 
skill  and  knowledge  requirements,  learning  times,  and  other  factors  essential  for  optimal  AFS 
structuring  decisions.  In  principle,  the  requirement  for  more  specific  task  data  can  be 
approach^  from  "past  and  frture"  directions  as  recommended  in  the  first  report  of  this  series 
ODriskill,  Boyle,  &  Garcia,  1986). 

Maintenance  data  from  the  MDGS  or  Logistics  Support  Analysis  Record  (LSAR) 
information  for  emerging  weapon  systems  and  equipment,  can  be  mapped  to  past 
occupational  survey  data.  This  mapping  will  satisfy  task  specificity  requirements  from 
historical  data.  The  mapping  can  provide  the  level  of  specifidty  desired  at  both  the  system 
and  component  level.  It  is  resource  intensive  and  time  consuming  to  cross-walk  occupational 
survey  tasks  and  data  to  other  maintenance  data  sources  manually.  Thus  there  is  a  need  for 
a  less  costly  mapping  process. 

Weapon  system-specific  task  data  from  future  occupational  surveys  could  be  obtained 
more  effidently  if  the  tasks  in  Job  Inventories  used  to  collect  job  data  are  based  on  weapon 
system-specific  task  data  in  MDGS  (or  LSAR  for  emerging  systems)  or  are  precoded  to 
\^Gs.  Detailed  component,  subsystem,  and  system  information  at  each  weapon  system 
WUG  nomenclature  level  can  be  provided.  Methods  for  developing  these  kinds  of  tasks  need 
to  be  identified  and  tested. 

In  addition  to  providing  more  specific  job  data  for  use  in  the  weapon  system 
acquisition  process,  improvement  of  inventory  development  technology  can  be  antidpated. 
The  presrait  invaitory  development  technology  was  developed  during  the  mid-1960s  when 
Air  Force  Specialty  (AFS)  structure  was  much  less  complicated,  especially  maintenance 
specialties  in  which  fewer  aircraft  and  less  equipment  were  common  place,  and  before  the 
multiplicity  of  manpower,  personnel,  and  training  uses  of  task  data  were  realized.  As  more 
weapon  systems  and  associated  equipmait  were  included  in  the  work  of  the  various  AFSs, 
inventory  development  and  survey  technology,  except  for  GODAP  capabilities,  did  not  keep 
pace,  '^ere  simply  have  been  no  techniques  for  accessing  and  utilizing  maintenance  task 
information  in  the  MDGS  or  the  LSAR,  which  can  be  extremely  useful  sources  of  data  for 
Job  Inventory  development. 

As  a  result  of  specialties  becoming  more  complex,  tasks  became  more  generic,  except 
for  the  weapon  system-specific  AFSs  for  the  more  recent  weapon  systems  (F-lll,  F-15,  F- 
16,  B-1).  Even  these  dedicated  AFSs  are  limited  to  select  subsystems  (e.g.,  avionics)  and 
much  of  the  work  on  the  remaining  systems  (e.g.,  fuels,  environmental,  electrical)  has  been 
incorporated  in  general  specialties  covering  all  aircraft. 

A  problem  is  the  scope  of  generic  AFS.  Table  1  shows  the  number  of  AFS  for  which 
generic  task  statements  must  be  generated  during  job  inventory  development  for  weapon 
systems,  missiles,  and  communications-electronic  equipment.  As  can  be  seen,  the  number 
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of  generic  AFS  is  considerably  larger  than  system-q)ecific  AFS. 


Table  1.  Number  of  Generic  Maintenance  Air  Force  Specialties 
Kind  of  AFS 


E* 


Generic  maintenance  (all  aircraft  covered,  no  shreds) 

Generic  AFS,  command  specific  shreds 
Generic  plus  aircraft  specific  shreds 

Other  generic  AFS  for  missiles  and  communications-electronic 
Subtotal  of  generic  AFS 


Aircraft  specific 

Total  maintenance  AFS 

♦Numbers  obtained  from  counting  AFS  on  the  Airman  Classification  Chart,  dated- 
31  October  1988. 


As  a  general  rule,  under  present  OSM  inventory  development  procedures,  specific 
tagVg  are  easily  described  for  equipment,  missile,  or  aircraft  specialties  in  which  only  one 
or  a  very  few  equipment  items  are  included.  Some  of  the  AFSs  resulting  from  RIVET 
WORKFORCE  have  been  formed  with  this  idea  in  mind.  Use  of  MDCS  task  information 
available  for  each  of  these  weapon  systems  should  make  specific  task  lists  even  more  easily 
developed. 

Generic  tasks  are  the  only  inventory  development  option  under  the  present  OSM 
technology,  for  AFS  having  multiple  equipment,  missiles,  and  weapons  systems.  The 
generality  of  task  statements,  however,  can  cause  job  incumbents  to  misunderstand  the  intent 
of  the  statement  as  well  as  result  in  the  development  of  tasks  which  overlap  with  other  tasks 
in  the  inventory. 

Furthermore,  research  value  exists  in  the  potential  for  identifying  a  methodology  that 
is  the  basis  for  the  development  of  Job  Inventories  for  nonmaintenance  specialties.  These 
specialties  usually  are  described  by  generic  tasks  that  provide  much  less  information  than 
more  q)ecific  tasks  could.  The  development  of  a  taxonomy  describing  the  structure  of  a 
specialty  and  locating  and  using  automated  data  bases  with  information  relevant  to  the  work 
of  a  specialty  would  be  extremely  useful  for  developing  task  lists  for  nonmaintenance 
specialties. 
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Status  of  Research  on  Weapon  Svstem-Speciiic  Tasks 


Two  previous  r^rts  have  addressed  the  question  of  mapping  OSM  data  and  other 
maintenance  data  sources  and  developing  weapon  system-q)ecific  task  lists.  In  the  first 
repoit  (Driskill,  Boyle,  &  Garcia,  1986),  occupational  survey  (OSM)  data,  MDCS  data,  the 
LSAR,  and  Logistic  Composite  Model  ^COM)  input  and  output  were  described  in  detail, 
and  the  compatibility  of  the  data  for  crosswaUdng  among  the  various  sources  was  assessed. 
Briefly,  each  of  the  data  sources  provide  maintenance  task  data,  although  the  LSAR  is  from 
a  developmental  perspective.  The  data  in  the  systems  are  complementary.  Unfortunately 
orosswalking  between  OSM  data  and  the  other  sources  is  limited  to  verbal  linkages.  When 
the  first  research  was  reported,  the  only  linkage  possibility  was  manual  mapping  of  OSM  to 
MDCS  and  the  other  maintenance  data.  The  crosswalking  between  MDCS  and  LCOM,  that 
occurs  in  the  preparation  of  input  data  to  the  LCOM  in  which  WUCs  from  the  MDCS  are 
key  data  elements,  was  identified.  Although  similarity  exists  between  LSAR  and  the  MDCS, 
some  problems  in  mapping  LSAR  to  MDCS  were  identified,  primarily  having  to  do  with 
variations  in  WUCs.  Considerable  manual  mapping  would  thus  be  required.  This  report 
recommended  semantic  mapping  as  a  possible  solution  and  estimated  that  perhaps  as  much 
as  60  percent  of  the  mapping  could  be  accomplished  using  a  microcomputer-based  semantic- 
assist^  analysis  methodology. 

The  second  report  (Driskill,  Weissmuller,  and  Staley,  1987),  had  two  objectives.  The 
first  objective  was  the  development  of  a  microcomputer-based  methodology  for  semantically 
matnhitig  tasks  in  the  OSM  data  base  with  the  tasks  in  the  MDCS.  Occupational  survey  tasks 
invariably  consist  of  a  verb  (an  action  taken)  and  an  object  (the  "thing"  acted  upon  by  the 
verb).  The  MDCS  tasks  consist  of  the  verbal  translation  of  the  action  taken  codes  (ATC) 
used  in  the  maintenance  of  various  WUC  equipment  items  of  a  given  weapon  system  (the 
objects  of  the  actions  tal^n).  MDCS  actions  taken  generally  parallel  the  verbs  employed  in 
OSM  inventory  tasks  while  the  MDCS  WUC  equipment  identification  is  semantically  similar 
to  and  parallel  the  task-verb  objects  in  inventory  tasks.  As  inventory  tasks  become  more 
generic  (that  is,  tasks  intended  to  describe  the  work  performed  on  multiple  systems),  OSM 
inventory  task  objects  and  WUC  equipment  identification  terminology  often  become  less 
semantically  similar. 

In  the  Semantic-Assisted  Analysis  Technology  (SAAT)  prototy^  (briefly  described 
in  a  later  section)  developed  to  accomplish  the  mapping,  MDCS  maintenance  data  for  a 
weapon  system  and  OSM  data  for  a  given  AFS  are  downloaded  to  a  microcomputer.  A 
series  of  programs  are  then  employed  to  produce  a  map  of  semantically  similar  OSM  and 
MDCS  tasks.  Subject  matter  experts  (SMEs)  verify  and  edit  the  microcomputer-generated 
matching  and  map  any  unmatched  items. 

The  initial  tryout  of  the  SAAT  prototype  involved  four  AFSs  which  perform  on- 
equipment  maintenance  on  the  F-16.  This  SAAT  mapping  eliminated  from  65  to  85  percent 
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of  the  maniial  matching  that  would  otherwise  be  required.  This  exceeded  the  initial  estimate 
of  the  SAAT  efficiency  by  5-25  percent.  Further,  SMEs  required  only  about  two  hours  to 
complete  the  mapping,  thus  reducing  the  resource  intensiveness  that  would  be  associated  with 
a  completely  manual  process.  Three  of  the  four  AFSs  covered  three  weapon  systems  (F-1 1 1 , 
F-15,  F-16)  while  the  fourth  AFS  was  generic  covering  all  aircraft.  Of  particular  interest 
was  the  fact  that  many  of  the  unmatched  items  for  each  AFS  resulted  from  the  genaic 
statements  used  to  describe  the  OSM  inventory  task-verb  objects. 

The  second  objective  of  the  study  was  the  development  of  a  prototype  microcomputer 
system  to  produce  a  component-specific  OSM  task  list  based  on  MDCS  WUC  data  for  a 
single  weapon  system.  The  ease  of  producing  such  a  task  list,  using  a  microcomputer-based 
methodology,  for  an  AFS  was  demonstrated.  An  inventory  of  tasks  for  an  AFS  whose 
incumbents  perform  on-equipment  maintenance  on  the  F-16  was  produced  from  MDCS 
maintenance  data  for  use  by  the  SAAT.  A  series  of  programs  produces  a  task  list  consisting 
of  the  MDCS  ATC  (task  verb)  and  the  WUC  equipment  identification  items  (task  object)  on 
which  the  actions  are  taken.  Included  for  each  item  are  statistical  data  showing  such 
information  as  the  frequency  of  the  action  taken  on  the  WUC  equipment  identification  object, 
the  average  time,  and  the  average  crew  size.  No  tasks  were  produced  from  the  Support 
General  or  Inspection  Codes,  although  such  tasks  can  be  used  in  the  same  set  of  programs. 

Obviously,  such  a  task  list  represents  only  the  maintenance  tasks  performed,  which 
is  a  part  of  the  total  tasks  performed  by  personnel  in  the  AFS.  Management,  supervisory, 
and  training  tasks  are  not  available  in  the  MDCS  and  must  be  added  to  the  Job  Inventory. 
Similar  system  and  component-specific  task  lists  for  any  single  weapon  system  (or  missile 
or  communication  electronics  systems)  can  be  product  for  any  maintenance  (e.g.,  on- 
equipment,  off-equipment,  on-equipment  non-airbome,  etc.)  performed  on  the  system.  The 
information  pertaining  to  toe  type  of  maintenances,  such  as  flightline  (on-equipment)  or  shop 
(off-equipment),  personnel  from  specific  AFSs  perform  is  essential  in  obtaining  maintenance 
data  tapes  from  toe  MDCS.  Further,  it  is  imperative  that  toe  MDCS  data  for  aircraft  be 
provided  on  separate  data  tapes  because  of  the  difficulty  encountered  in  sorting  toe 
maintenance  data  when  more  than  one  aircraft  data  set  are  included  on  a  single  tape.  On  toe 
MDCS  data  tapes,  aircraft  identification  is  readily  identifiable  only  for  maintenance 
performed  on  the  flightline. 

A  method  for  developing  Job  Inventories  for  generic  AFSs  covering  multiple  weapon 
systems  (or  communications-electronics  equipment  or  missile  systems)  from  MDCS  WUCs 
has  not  yet  been  developed.  Development  of  these  inventories  is  a  far  more  difficult 
endeavor  and  will  be  covered  in  later  sections  of  this  report.  Results  of  a  further  tryout  of 
toe  production  of  single  system-specific  on-equipment  task  lists  from  WUCs  for  two  AFSs 
working  on  toe  F-15  and  one  AFS  on  toe  B-IB  will  also  be  discussed.  But,  first,  in  toe 
following  sections,  results  of  further  tryout  of  toe  SAAT  on  two  AFSs  performing  on- 
equipment  maintenance  on  toe  F-15  will  be  described. 
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ffl.  SEMANnC-ASSKTED  ANALYSIS  TECHNOLOGY 


This  section  briefly  describes  the  Semantic-Assisted  Analysis  methodology  and 
presents  a  discussion  of  important  steps  requiring  analyst  intervention  in  mapping  MDCS 
action  taken  and  work  unit  code  equipment  items  to  occupational  survey  tasks  and  data. 


Description 

Eiq)erience  as  well  as  previous  research  (Wagner,  1986)  reveals  that  SMEs  can 
manually  map  much  of  the  MDCS  data  for  an  aircraft  to  many  of  the  tasks  in  an  occupational 
survey  parallel  to  the  MDCS  data.  Manual  mapping  is  resource  intensive  and  time 
consuming.  The  Semantic-assisted  Analysis  Technology  was  developed  on  the  premise  that 
a  microcomputer  methodology  could  be  developed  to  reduce  the  worldoad  associated  with 
the  manual  mapping. 

In  any  responsible  data  base  management  system,  data  are  collected  at  what  is 
considered  to  be  a  reasonable  level  of  detail  for  the  typical  user.  In  the  development  of  the 
MDCS,  logistical  users  were  identified,  thus  the  data  in  the  system  has  to  do  with  the 
reliability  and  maintainability  of  eqmpment.  For  the  OSM,  p^sonnel  and  training  ftmctions 
were  envisioned  as  major  users  in  its  design  and  development,  and  the  resulting  data  are 
collected  and  reported  at  a  much  broader  but  generally  acceptable  level  of  detail  than  in  the 
MDCS.  Time  and  cost  typically  preclude  collecting  survey  data  at  the  lowest  level  of  detail 
desired  by  specific  categories  of  users.  Frequently  other  data  base  users  are  identified  after 
design,  development,  and  implementation,  as  in  the  case  of  the  mapping  survey  and  MDCS 
data.  The  problem,  then,  lies  in  developing  some  methodology  which  allows  description  of 
the  data  fields  to  be  analyzed,  decomposed,  cross-referenced,  and  re-synthesized  to  yield 
estimates  of  data  for  different  structures  or  levels  of  detail. 

A  tey  step  in  this  process,  particularly  where  different  data  bases  are  developed  for 
different  purposes  and  users,  is  a  semantic  interpretation  of  the  descriptions  themselves.  In 
this  context,  "semantic  interpretation"  means  developing  a  knowledge-structure  (or  schemata) 
which  relates  and  organizes  concepts  into  one  or  more  taxonomies  useful  for  the  purposes 
at  hand.  The  problem  here  lies  in  the  fact  that  the  same  physical  workspace  may  be 
described  by  very  dissimilar  taxonomies  ("World  Views")  by  Afferent  users  (i.e.,  from 
different  functional  perspectives  and  goals).  Each  World  View  is  composed  of  concepts 
which  may  be  either  references  to  physical  objects  (objective)  or  references  to  perceived 
relationships  or  states  (theoretical).  In  addition,  concepts  are  conveyed  by  totens,  such  as 
words,  phrases,  abbreviations,  or  symbols,  giving  rise  to  confusion  when  different  tokens 
(synonyms)  can  be  used  to  indicate  the  same  concept,  or  when  a  given  token  can  mean  or 
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imply  different  concepts  depending  upon  its  context  (an  equivocation).  These  types  of 
interpretations  can  be  made  only  by  trained  SMEs. 

Two  mapping  approaches  could  be  taken.  The  first,  and  most  labor  intensive,  is  to 
have  SMEs  meet,  review  the  array  of  data,  propose  a  taxonomy,  and,  for  each  data  element, 
map  it  onto  or  associate  it  with  aitries  in  the  taxonomy.  The  second  approach  is  to  pre- 
process  all  written  materials  by  computer  and  have  intelligent  software  create  a  tentative 
structure  and  mapping  before  the  SMEs  arrive. 

The  second  approach  has  many  advantages.  First,  the  structure  created  will  be 
rq)roducible  with  the  rules  used  clearly  defined.  This  step  is  facilitated  if  an  existing 
structure  is  selected  as  a  starting  point,  even  if  it  is  not  the  final  structure  employed. 
Second,  this  approach  reduces  the  fatigue  factor  on  the  SME  by  performing  the  mechanical 
work  they  would  have  to  do  under  the  purely  manual  approach.  Third,  computer  time  is  less 
e3q)ensive  than  SME  time.  Fourth,  the  computer  will  make  mistakes  in  the  mapping,  but  the 
SME’s  first  job  is  to  redline  those  errors  which  quickly  demonstrates  their  value  to  the 
project  and  reinforces  their  active  participation  in  this  high-level  decision-making.  Fifth, 
SME  recognition  skills  are  better  than  their  recall  skills  and,  hence,  progress  is  expedited  in 
the  iniriat  phase.  Finally,  because  they  clearly  have  in  mind  what  the  objectives  are  (from 
the  previous  step),  the  SME  can  do  a  better  job  when  they  inspect  "highly  valued"  items 
which  are  NOT  currently  mapped  and  allocate  them  to  their  proper  places  in  the  hierarchy. 

The  second  approach  is  embodied  in  the  Semantic-assisted  Analysis  Technology,  In 
recognition  of  the  inherent  differences  in  the  internal  structures  of  the  descriptions  of  OSM 
and  MDCS  tasks,  automated  cross-referencing  in  the  development  of  the  initial  SAAT  is 
based  on  tey  words  or  phrases  (An  enhancement  which  will  incorporate  "values"  derived 
firom  OSM  and  MDCS  data  for  these  words  or  phrases  will  be  proposed  in  a  later  section 
as  a  means  of  making  SAAT  more  powerful  and  producing  even  better  matches).  While 
linguistic  analysis  is  possible  on  OSM  task  statements,  the  MDCS  system  fundamentally 
sq)arates  verbs  (actions  taken)  from  objects  and  drops  all  references  to  actors  (in  OSM  tasks, 
the  actor  is  the  assumed  word  "I"  preceding  each  task  statement). 

Although  processing  of  MDCS  data  is  simplified,  because  the  data  are  barren  of 
theoretical  content  other  than  in  verb  choice,  processing  is  confounded  by  the  hierarchical 
nature  of  MDCS  data  and  the  lack  of  standardization  in  word  order  and  abbreviations  in 
WUCs.  In  the  MDCS,  each  WUC  is  a  hierarchically  arranged  5-character  code.  Each 
aircraft  system  has  a  designated  2-character  code  which  comprises  the  first  two  characters 
of  the  WUC  (e.g.,  51 — ,  Instruments;  72 — ,  Radar  Navigation).  The  2-character  code  is 
invariant  across  aircraft.  The  third  character  designates  the  subsystem;  the  fourth  character, 
a  sub-subsystem;  and  the  fifth  character,  a  sub-subsystem  component  part.  On  the  F-15,  for 
example,  WUC  51EAB  represents  Instruments  (51),  Air  Data  Systems  (E),  Computer,  Air 
Data  AN/ASK-6  (A),  CONVERTER,  Frequency  to  Digital  (B),  The  third,  fourth,  and  fifth 
characters  of  the  WUC  vary  across  aircraft.  Because  of  the  hierarchial  nature  of  the  MDCS 


8 


information,  an  equipmrat  item  defined  by  the  fifth  character  of  a  WUC  is  inherited  by  the 
fourth  character,  and  so  on,  it  is  the  starting  point  in  the  mapping  process.  An  item  defined 
at  the  third,  fourth,  or  fifth  character  level  has  no  meaning  if  it  is  not  "read"  in  the  context 
of  the  preceding  characters.  In  the  WUC  illustrated  above,  the  fifth  level  item 
CONVERTER,  Frequency  to  Digital,  can  only  be  interpreted  in  light  of  the  preceding 
charactCTS  showing  that  the  Converter  is  a  part  of  the  Air  Data  Computer  in  the  Air  Data 
Systems  of  the  F-15  ftistruments  System. 


Semantic-AssLsted  Analysis  Technolo£V.  Sources.  Processm£.  and  Output 

Figure  1  provides  a  conceptual  description  of  the  sources  and  data  di^layed  in  the 
final  mapping  of  OSM  and  MDCS  data  and  shows  the  two  sources  of  data  that  are  mapped. 
Presently  SAAT,  MDCS  data  are  obtained  from  two  sources:  the  B-4  Master  File,  which 
contains  a  complete  listing  of  the  WUCs  and  Equipment  Idratification  for  a  given  aircraft 
(or  missile  or  communications-electronic  equipment);  and  a  MDCS  data  tape  containing  the 
actual  aggregated  force-wide  maintenance  records  (as  reported  on-site  by  workers  using  Core 
Automated  Maintenance  System  (CAMS)  or  manually  on  AF  Form  349)  for  the  givai 
aircraft  for  a  specified  period  of  time.  The  MDCS  Equipment  Id^tification  (that  is,  the 
names  of  the  equipment  items)  is  in  abbreviated  form,  the  description  of  the  equipment 
names  being  limited  to  a  total  of  19  characters.  As  a  part  of  the  processing  requirements, 
the  abbreviated  titles  are  fiiUy  expanded  to  reflect  the  complete  title.  OSM  data  are 
retrieved  from  the  occupation^  survey  data  base  for  the  given  AFS.  All  sets  of  data  are 
downloaded  from  the  Air  Force  Human  Resource  Laboratory  (AFHRL)  Sperry  1100/82  to 
an  AT  compatible  microcomputer. 

In  addition,  to  obtaining  the  MDCS  data  tapes,  the  analyst  should  also  obtain  a  copy 
of  the  Work  Unit  Code  Manual  for  the  aircraft  of  interest  and  a  copy  of  the  USAF  Job 
Inventory  for  the  AFS  of  interest.  An  aircraft  WUC  Manual  is  a  Technical  Manual  under 
the  -06  Technical  Order  series,  and  is  available  from  the  Air  Logistics  Center  having 
maintenance  responsibility  for  the  aircraft  of  interest.  The  WUC  Manual  is  extremely  useful 
for  reference  purposes  throughout  the  mapping  process.  In  many  cases,  the  abbreviated  titles 
in  the  B-4  Master  File  are  provided  in  more  fiilly-speUed  out  form,  thus  facilitating  the 
expansion  of  the  titles  necessary  in  the  SAAT. 

A  third  information  source,  which  provides  a  matching  of  AFS  with  the  WUC  used 
in  maintaining  the  aircraft  of  interest,  is  required  as  a  basis  for  the  mapping  of  the  data 
bases.  To  date,  the  Percent  Contribution  Report  provided  by  HqTAC/XPMS  has  been  the 
source,  since  Tactical  Air  Command  aircraft  have  been  used  in  testing  SAAT.  This 
document,  an  output  of  the  Logistics  Composite  Model  (LCOh^  used  in  manpower  planning, 
usually  provides  the  WUC  to  the  third  character  level  for  an  AFS  for  those  WUCs  for  which 
an  AFS  is  to  receive  workload  credit. 
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MDC  System 


I 


OSM  System 


Data  I  Descriptors 


Work  Unit  Code 
Action  Taken  Code 
Crew  Size 
Time  Recorded 


Word  Unit  Code  with  19 
Character  Oescription 

Action  Taken  Code  with 
Explanations  in  the 
Handbook 


- Action  1 - ^ 

WUC  Text  Data 
WUC  Text  Data  4 


- Action  2 - 

WUC  Text  Data 


Data 


Job  Incumbent 
Summary  Data  for 
Selected  Groups: 

%  Time  Spent 
%  Members  Perf. 


Subject-Matter 
Experts  ratings  of 
Learning  Difficulty  or 
Training  Emphasis 


Descriptors 


Definitions  of  background 
questions  fimited  to  one 
line  of  66  characters. 

Task  Statements  fimited  to 
1 5  fines  of  68  characters 
In  the  form  of  <T> 
verb  noun—  phrase. 


OSR  Task  #01  „Data.„ 
WUC  Text  Data 
WUC  Text  Data 

OSR  Task  #02  —Data... 
WUC  Text  Data 


Figare_i.  Data  Sources. 
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At  present,  the  only  readily  available  source  of  WUC-AFS  crosswalk  information 
resides  at  Aeronautical  Systems  Division,  Tactical  Air  Command,  Military  Airlift  Command, 
and  Strategic  Air  Command  LCOM  agencies.  Table  2  shows  the  systems  for  which  the 
WUC-AFS  crosswalk  information  is  available  upon  request. 


Table  2.  Systems  for  Which  LCOM  WUC-AFS  Data  Are  Available 


Location 


Systems 


Aeronautical  Space  Division 
Military  Airlift  Command 
Strategic  Air  Command 
Tactical  Air  Command 


Advanced  Tactical  Fighter,  C-17,  LANTIKN 
C-130  E  and  H,  C-141,  C-5,  AC-130,  MC-130 
KC-135R 

F-15,  F-16,  F-4,  F-111,  E-3,  A-10 


(Information  extracted  from  AIR  FORCE  CROSSWALK,  1989,  p.  1-7) 


For  other  systems,  the  sources  at  present  are  limited  to  SME  interview  or  survey  or 
through  accessing  base-level  CAMS  data  at  bases  maintaining  the  aircraft  of  interest.  CAMs 
replaces  matuial  reporting  of  maintenance  work.  At  base  level,  CAMS  consolidates  the  AFS 
of  workers  with  maintenance  records  that  include  WUCs.  Approximately  40  percent  of  the 
Air  Force  bases,  for  which  funds  for  the  automated  system  have  not  been  allocated,  are  still 
operating  under  the  manual  system  of  reporting  maintenance,  using  AF  Form  349.  By 
approximately  the  end  of  FY  1991,  aU  bases  are  expected  to  have  converted  to  CAMS 
reporting.  The  proposed  Air  Force  CROSSWALK  Project,  if  completed,  could  provide 
immediate  access  to  the  required  WUC-AFS  mapping  information,  since  it  is  proposed  that 
a  large  part  the  Project  will  rely  on  base-level  CAMS  reporting  (AIR  FORCE 
CROSSWALK,  1989).  Information  from  surveys  or  interviews  and  CAMS  are  potentially 
the  best  sources  of  information  for  use  in  SAAT  because  WUC-AFS  information  is  more 
precise  than  is  found  in  the  LCOM  information.  For  management  engineering  use,  WUC 
identification  at  the  3-digit  level  is  adequate,  but  is  inadequate  for  SAAT  mapping.  The 
importance  of  precise  WUC-AFS  crosswalk  data  can  be  noted  in  a  later  section  reporting  the 
results  of  the  latest  mapping  of  OSM  and  MDCS  data. 

The  actual  SAAT  mapping  is  based  on  an  expansion  of  the  abbreviations  used  in  the 
19-character  description  (equipment  identification)  of  the  WUCs  in  the  B-4  file  and  the  verb 
objects  in  the  OSM  task  statements.  MDCS  and  OSM  statements  are  cross-matched  based 
upon  the  use  of  common  words,  phrases,  or  concepts  (tokens). 
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Cross-match  candidates  are  evaluated  within  the  context  of  knowing  the  information 
value  of  the  common  words,  phrase,  or  concepts  based  on  their  frequency  of  occurrence. 
In  the  processing,  all  related  words,  phrases,  or  concepts  are  rq)laced  with  a  common  toten, 
requiring  analyst  intervention  in  resolution  of  a  small  number  of  items  that  the 
microcomputer  programs  cannot  resolve.  The  number  requiring  intervention  varies  by  WUC 
by  aircraft,  because  of  the  kind  of  abbreviations  employed.  Problems  encountered  in  this 
replacement  process  include  abbreviations,  acronyms,  hyphenations,  parenthetical 
e}q)ressions,  compound  words,  and  contexts. 

Once  the  related  words  are  replaced,  the  next  stq)  requires  determination  of  the 
information  value  of  common  tokens.  Information  value  is  established  by  identifying  a 
subset  of  statements  of  interest  and  comparing  token  occurrence  frequency  inside  the  set  to 
their  frequency  outside  the  set. 

The  SAAT  mapping  process  is  described  in  detail  in  the  SAAT  documentation 
(Driskill,  et  al. ,  1987).  This  documentation  provides  instruction  for  the  computer  technician, 
and  indicates  where  analyst  intervention  is  required  to  make  judgments  about  such  items  as 
abbreviations,  hyphenations,  and  plurals.  In  these  instances,  the  analyst  is  provided 
instructions  about  the  judgm^its  to  be  made.  In  most  cases,  the  intervention  consists  of 
verifying  changes  suggested  by  the  SAAT  software. 

Most  of  the  work  involved  occurs  in  the  preparation  of  the  OSM  tasks  and  MDCS 
"tasks"  for  matching.  It  is  important  to  remember  in  this  preparation  effort  that  the  matching 
is  based  on  the  existence  of  tokens  common  to  the  two  data  bases.  The  preparation  of  each 
of  the  Hafa  bases,  therefore,  focuses  on  the  development  of  tokens  (from  the  words,  phrases, 
and  concepts  contained  in  the  data  bases)  that  wiU  ensure  the  commonality  necessary  for 
matching  to  occur.  For  example,  a  token  like  ACFT  in  the  MDCS  will  not  match  the  token 
AIRCRAFT  in  an  OSM  task  statement;  ANGLE  OF  ATTACK  will  not  match  ANGLE-OF- 
ATTACK:  nor  will  GYROSCOPE  match  GYROSCOPES.  The  objective,  however,  is  not 
to  force  commonality,  but  to  prepare  the  totens  in  such  a  way  that  token  commonality  will 
be  identified  where  it  exists. 

The  SAAT  includes  microcomputer  programs  which  accomplish  the  following  actions: 

1.  Eliminating  abbreviations 

2.  Eliminating  dangling  hyphens 

3.  Correcting  misspelled  words 

4.  Recovering  missing  outline  levels  (missing  WUCs  and  equipment  identifications) 

5.  Substituting  acronym  definitions  at  every  occurrence  of  an  acronym 
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6.  FJiminating  superfluous  hyphens  and  parenthetical  phrases  in  the  text 

7.  Identifying  and  replacing  plural  forms  of  words 

8.  Manually  identifying  semantically  parallel  terms  from  microcomputer  output 
provided  for  this  purpose 

9.  Automatically  collapsing  references  to  semantically  parallel  terms. 

The  elimination  of  abbreviations  used  in  the  MDCS  data  base  requires  most  of  the 
time  an  analyst  spends  in  the  SAAT  process.  This  step  is  crucial.  The  process  expands  the 
abbreviations  used  in  the  B-4  Master  File  WUC  Equipment  Identification  (names  of 
equipmait  entities)  to  their  full  titles.  Each  entity  expanded  in  this  step  becomes  a  token 
used  in  subsequent  mapping.  In  Figure  2,  for  example,  the  abbreviated  entity  ACFT  is 
expanded  to  AIRCRAFT.  It  is  especially  important  that  each  expansion  is  made  in  the 
context  of  the  WUC  in  which  the  abbreviation  appears.  Most  of  the  expansion  difficulty  lies 
in  the  lack  of  consistency  of  the  abbreviations  used  across  WUC  Manual  equipment 
identifications. 

As  previously  indicated,  sources  for  expanding  the  abbreviations  are  limited. 
Unfortunately,  personnel  who  maintain  the  WUC  manual  for  various  aircraft  are  unaware 
of  any  fully  expanded,  unabbreviated  listing  of  the  WUC  Equipment  Identification  items 
available,  Thus,  at  present  the  best  sources  for  e^anding  the  abbreviations  are  the  WUC 


Manual  (-06),  which  also  contains  many  abbreviations,  and  personal  knowledge-either  on 
the  part  of  the  analyst  or  through  use  of  SMEs.  Telephone  contacts  with  SMEs  have  beai 
especially  helpful  to  resolve  abbreviations  that  the  analyst  could  not  decode. 

Work  Unit  Code  Manual  Abbreviation 

Expanded  Form 

ACFT 

AIRCRAFT 

ACT 

ACTUATOR 

ADAPT 

ADAPTER 

ADI 

ATTITUDE  DIRECnONAL 
INDICATOR 

AIRSPED 

AIRSPEED 

ANL 

ANALOG 

AOA 

ANGLE  OF  ATTACK 

Figure  2.  Example  of  Abbreviated  Work  Unit  Code  Expansion. 


Although  expansion  of  the  abbreviations  is  the  most  time  consuming  step  in  the 
process,  once  the  abbreviated  equipment  identifications  in  a  B-4  file  for  an  aircraft  are 
expanded  for  use  in  mapping  to  task  statements  for  one  AFS,  the  process  does  not  need  to 
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be  completed  again  when  mapping  other  AFS  to  the  aircraft.  Enlran^Mts  will 
suggest^  in  a  laL  section  of  this  report  that  wiU  reduce  the  work  associated  with  eitpandmg 

the  WUC  abbreviations  effort. 

Before  matching  can  occur,  four  data  screens  must  be  appUed  to  the  OSM  ^ 
data.  The  first  screen,  on  OSM  data,  is  based  on  the  AFS  job  desmpfion  to  be  us^.  This 
screen  is  necessary  to  identify  the  tasks  performed  by  mcumbents  of  the  target  AFS.  A 
number  of  opti^are  available  for  selecting  the  OSM  job  description  to  ^  “ 

descriptions  based  on  time  in  service,  skill  level,  job  group  (ca^  or  task  clustemg)  major 
command  of  the  personnel  performing  maintenance  on  the  aircraft  of  mterest,  and  tas 
data  Since  the  application  of  the  SAAT  to  date  has  been  for  AFS  covemg  multiple 
weapons  systems,  the  screen  applied  has  been  based  on  the  perwntage  of  mc^bente 
nerforming  tasks  only  on  the  aircraft  of  interest  (e.g.,  F-15)  and  who  possess  5-skill  Iwe 
duty  AFS.  In  addition,  tasks  were  screened  to  eliminate  those  for  which  no  WUC  could  be 
anticipated  (e.g.,  duties  pertaining  to  directing  and  training)  and  for  which  the  percentage  of 
personnel  performing  was  15  percent  or  less. 

Following  the  appHcation  of  this  screen,  two  additional  screens  are  appHed  to  toe 
OSM  tasks  First,  through  use  of  one  of  toe  microcomputer  programs  mcorporated  m  toe 
b^g  ^th  verbs  that  cannot  be  related  to  WUC  action  ^ 

eliminated  (e.g.,  review,  write,  complete).  Next,  tasks  conta^g  fte 
toe  object  was  preceded  with  different  verbs,  were  consohdated.  Object  consohdation 
eUminates  toe  requirement  for  SMEs  to  review  tasks  comprised  of  a  sen^  of  tasks  on  toe 
same  object  suc^as  isolating  malfunctions,  operationally  checking,  adjusting,  “ 

replacing,  or  calibrating  (same  object).  Instead,  SMEs  cm  respond  to  a  single  ta 
rraresenting  toe  action  taken  on  a  given  object.  Sigmficant  SME  fame  is  sav^.  At 
potot,  the  Mtion  taken  is  unimportant;  this  information  will  be  supphed  from  actual 

maintenance  records  in  later  mapping. 

The  screen  applied  to  the  MDCS  data  is  based  on  the  ^UC-AFS  crosswalk 

information,  and  is  necessary  to  narrow  toe  scope  of  toe  Ss^cro^alk 

personnel  of  toe  AFS  of  interest.  The  greater  toe  accuracy  of  toe  WUC-AFS  crosswalk 

screen,  toe  greater  toe  SAAT  mapping  potential. 

Following  toe  appHcation  of  screens,  SAAT  mapping  can  be  accompHsh^.  The 
product  of  the  mapping  is  a  printing  of  mapped  OSM  and  m  a 

^e  booklet  contains  WUCs  and  associated  nomenclature  mapped  bei^eath  toe  OSM  ^ 
SMEs  then  use  toe  booklet  to  verify  and  edit  toe  microcomputer  generated  matchmg  of  OSM 
and  MDCS  tasks  (see  Figure  3  for  a  sample  page). 
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After  the  SMEs  review,  a  final  listing  of  the  mapped  information  (Figure  4)  is 
produced.  This  list  contains  each  task  in  the  job  inventory  used,  and  shows  the  percentage 
of  incumbents  performing,  percentage  of  time  spent  in  performance,  and  task  difficulty  and 
training  emphasis  mean  ratings.  Immediately  below  each  task  is  a  listing  of  the  ATC,  and 
the  WUC  and  Equipment  Identification  of  each  item  that  job  incumbents  work  on  in 
performing  the  task.  MDCS  summary  data  can  also  be  shown  for  such  items  as  frequency 
of  maintenance,  average  maintenance  time,  and  crew  size. 

The  entire  mapping  process  from  acquisition  of  data  tapes  and  downloading  to  a 
microcomputer  through  production  of  the  final  result  of  the  mapping,  is  detailed  in  the 
software  documentation.  Computer  technicians  with  experience  in  ASCII  CODAP  processing 
on  the  AFHRL  computer  system  can  follow  the  documentation.  Analyst  e^rience  with  task 
list  development  for  aircraft  maintenance  specialties  or  knowledge  of  these  specialties  is 
desirable. 
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IV.  MAPPING  MDCS  AND  OSM  DATA  FOR  AFS  452X1B 
and  454X3  WORKING  ON  THE  F-15 


Further  testing  of  the  SAAT  was  accomplished  by  mapping  OSM  tasks  and  data  to 
MDCS  data  for  two  AFS  maintaining  the  F-15.  The  overall  objectives  were,  first,  reporting 
of  results  of  mapping  OSM  and  MDCS  data  for  AFS  452X2B,  F-16  Avionics  Instrument, 
and  AFS  454X3,  Aircraft  Fuel  Systems.  Second,  enhancements  of  the  SAAT  were  to  be 
identified  for  future  development. 


OSM-MDCS  Data  Manning 

Several  questions  arise  concerning  the  mapping  of  MDCS-WUC  and  OSM  data.  They 
include: 

1 .  What  is  the  accuracy  of  SAAT  mapping  of  OSM  and  MDCS  task  data  and  how 
much  time  is  required  for  SMEs  to  verify  and  edit  the  SAT  mapping? 

2.  What  kinds  of  tasks  present  problems  for  SAAT  mapping? 

3.  What  is  the  SME  interrater  agreement  on  verification  and  editing? 

4.  What  are  the  primary  problems  encountered  by  SMEs? 

5.  How  accurate  is  the  WUC-AFS  crosswalk  screening  data  obtained  from  the 
Percent  Contribution  Report  from  the  LCOM  system? 

6.  What  enhancements  of  the  SAAT  will  produce  more  accurate  mapping  and 
reduce  processing  and  SME  verification  worldoad? 

In  the  following  paragraphs,  the  results  of  SME  verification  and  editing  of  data 
mapped  for  the  two  AFS  chosen,  as  well  as  observations  made  in  the  application  of  the 
SAAT,  are  described. 

1 .  Mapping  Accuracy.  To  verify  the  SAT  mapping,  two  SMEs  from  AFS  452X1B 
and  three  SMEs  from  AFS  454X3  working  on  the  F15A  at  Holloman  AFB  were 
utilized.  Separate  sessions  were  employed  for  each  AFS.  In  the  previous  SME 
verification  of  mapping  for  the  F-16,  only  one  SME  from  each  AFS  was 
utilized.  The  first  session,  involving  the  two  AFS  422X1B  SMEs,  was  used 
partly  to  determine  the  best  method  of  administration  when  more  than  one  SME 
are  involved. 
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The  AFS  452X1B  SMEs  were  instructed  to  review  the  mapping,  checking  the 
appropriate  WUC  under  a  task,  and  to  write  in  any  WUC  that  was  not  listed.  While  they 
were  not  instructed  to  work  indq)endently,  the  implication  was  that  they  should  do  so. 
Independent  work  is  a  necessary  prerequisite  for  determining  interrater  agreement  among 
raters.  It  soon  became  apparrat,  however,  that  they  could  not  work  entirely  independently, 
because  they  consistently  encountered  problems  with  interpretation  of  task  statemaits.  As 
a  result,  the  SMEs  ratings  were  not  entirely  independent.  A  tenable  assumption  is  that 
respondents  to  job  surveys  also  have  problems  with  interpretation  of  task  statements. 

For  the  second  group  of  SMEs  (those  with  AFS  454X3),  similar  instructions  for 
verifying  the  SAAT  mapping  were  provided.  Again,  SMEs  were  not  instructed  to  work 
independently,  but  the  requirement  was  implied.  They  were,  however,  asked  to  complete 
the  first  half  of  the  mapping  (through  page  11  of  23  pages)  at  which  time  the  administrators 
would  check  what  they  had  done  before  they  would  continue.  This  process  was  used  so  that 
instructions  or  other  matters  could  be  clarified.  These  SMEs,  also  encountered  task 
interpretation  problems  and  asked  if  they  could  work  together.  This  request  was  granted  and 
the  remainder  of  the  list  (about  75  percent  remained  to  be  accomplished)  was  completed  by 
the  SMEs  with  considerable  discussion  of  the  meaning  of  many  of  the  tasks. 

Including  the  time  for  providing  administrative  instructions,  the  first  group  of  SMEs 
(AFS  452X1B)  required  one  hour  and  55  minutes  to  complete  the  verification  of  the  SAAT 
mapping.  The  three  AFS  454X3  SMEs  who  rated  more  than  one-third  more  tasks  than  did 
the  SMEs  for  the  other  AFS  required  two  hours  and  20  minutes  to  complete  the  verification. 
The  question  of  agreement  among  raters  is  discussed  below.  It  should  be  noted  that  in  the 
previous  test  of  SAAT  mapping  for  the  Fuels  Specialty  for  the  F-16,  the  SMEs  working 
alone  did  not  complete  the  verification  and  editing  within  the  three  hours  allotted  (DriskUl 
et  al.,  1987). 

Results  of  the  SME  verification  and  editing  are  shown  in  Table  3. 


Table  3»  Results  of  SME  Verification  of  SAAT  Mapping  by  Number  of  Tasks 


Task  Cateeorv 

AFS  452X1 
N  % 

AFS  454X3 
N  % 

N  tasks  screened  for  matching 

189 

100.0 

118 

100.0 

Not  performed  by  AFS 

7 

3.7 

7 

5.9 

Not  performed  on  F-15 

38 

20.1 

27 

22.9 

Not  WUC-equipment  tasks 

0 

0.0 

19 

16.1 

AFS-WUC  not  provided  by  LCOM  Percent 
Contribution  Report 

17 

9.0 

0 

0.0 

19 


Table  3.  (Concluded) 


AFS 

452X1 

AFS 

454X3 

Task  Cateeorv 

N 

% 

JL. 

Total  unmappable  tasks 

62 

32.8 

53 

44.9 

Total  mappable  tasks  remainine 

127 

67.2 

65 

55.0 

Tasks  matched,  no  WUC  write-ins 

95 

74.8 

55 

84.6 

Tasks  matched,  write-ins  added 

15 

11.8 

4 

6.2 

Tasks  not  matched,  write-ins  added 

17 

13.4 

6 

9.2 

Several  conclusions  may  be  drawn  from  this  table: 

a.  The  SAAT  successfully  maps  MDCS  data  to  OSM  data  for  a  large 
percentage  of  tasks  where  mapping  of  an  aircraft  WUC  is  possible. 
Seventy-five  percent  of  the  mappable  tasks  in  AFS  422X1B  were 
accurately  mapped  by  the  SAAT  without  any  SME  additions  of  WUCs. 
For  AFS  454X3,  85  percent  of  the  tasks  ware  mapped  without  any  WUCs 
being  added.  Overall,  of  the  WUC-mappable  tasks,  87  percent  of  the  AFS 
422X1B  tasks  were  at  least  partially  matched  by  the  SAAT.  The 
percentage  of  partial  match  was  91  for  AFS  454X3. 

Since  there  were  a  number  of  tasks  for  which  there  was  no  possibility  of  mapping 
WUCs  by  the  SAAT  or  by  the  SMEs,  SAAT  mapping  efficiency  was  computed  by 
comparing  the  number  of  matched  tasks  with  the  number  of  tasks  for  which  mapping  was 
possible.  The  total  number  of  tasks  screened  (based  on  percent  of  5-skill  level  incumbents 
working  on  the  F-15)  for  mapping  was  118  for  AFS  452XB  and  189  for  AFS  454X3.  Of 
the  total  tasks  screen^  for  each  AFS,  there  was  no  basis  for  SAAT  mapping  of  62  tasks  for 
AFS  452X1B  and  53  tasks  for  AFS  454X3.  The  following  criteria  were  applied  to  define 
whether  a  task  was  unmappable  by  SAAT: 

(1)  A  task  for  which  the  AFS  had  no  responsibility  for  performing  was 
not  mappable  because  the  WUC  screen  did  not  permit  WUCs  used 
by  the  other  AFS  to  enter  the  mapping  process. 

(2)  A  task  that  could  not  be  performed  on  the  F-15  was  not  mappable, 
because  the  WUC  screen  did  not  permit  use  of  WUCs  for  other 
aircraft. 

(3)  If  the  task  was  generically  described  with  no  equipment-specific 
object  (e.g..  Perform  red  dye  tests),  there  was  no  opportunity  for 
mapping  WUCs. 
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(4)  If  the  WUC-AFS  screen  did  not  include  WUCs  that  job  incumbents 
in  the  AFS  rqwrted,  because  of  inadequacy  of  the  WUC-AFS 
crosswalk  data  that  was  the  basis  of  the  screen,  no  WUC  was 
available  for  mapping. 

There  were  four  classes  of  mappable  tasks: 

(1)  Tasks  for  which  the  SAAT  mapped  the  acceptable  WUCs  without 
any  additions. 

(2)  Tasks  for  which  the  SAAT  mapped  acceptable  WUCs  but  the  SMEs 
added  other  WUCs. 

(3)  Tasks  for  which  all  of  the  WUCs  mapped  by  the  SAAT  were 
unacceptable  and  SMEs  added  correct  WUCs. 

(4)  Tasks  for  which  there  was  acceptable  mapping  by  SAAT,  but  SMEs 
added  WUCs  that  could  not  be  located  in  the  MDCS  B-4  Master 
File.  The  added  WUCs  are,  however,  included  in  WUC  Manual 
amendments  which  have  not  been  added  in  the  B-4  file. 

b.  A  large  number  of  OSM  tasks  could  not  be  used  for  WUC  mapping. 
Although  the  OSM  tasks  were  screened  to  provide  data  on  incumbents  who 
indicated  that  they  worked  only  on  the  F-15,  38  (20  percent)  of  the  tasks 
identified  for  AFS  452X3B  could  not  be  performed  on  the  F-15.  For  AFS 
454X3,  there  were  27  (23  percent)  such  tasks. 

In  addition,  19  (16  percent)  AFS  454X3  tasks  did  not  relate  to  any  WUC.  This 
number  is  not  unexpected,  because  the  Fuels  Specialty  is  one  in  which  the  methods, 
materials,  or  processes  of  work  are  more  important  than  the  actual  equipment  the  action  is 
taken  upon  (e.g.,  perform  red  dye  tests).  Inclusion  of  these  tasks  in  the  Job  Inventory  was 
done  to  reduce  the  number  of  tasks  in  the  inventory. 

c.  For  purposes  of  semantic  matching,  the  WUC-AFS  matching  contained  in 
the  LCOM  Percent  Contribution  Report  can  be  expected  to  be  inadequate 
for  at  least  some  specialties.  There  were  17  tasks  (9  percent)  of  the  AFS 
452X1B  tasks  for  which  WUCs  were  not  provided  by  the  LCOM  Percent 
Contribution  listing. 

2.  Problem  Tasks.  Aside  from  the  tasks  that  could  not  be  used  for  mapping, 
because  of  missing  WUCs  (for  whatever  reason),  there  were  tasks  for  which  the 
SAAT  did  not  map  any  acceptable  WUCs.  These  were  tasks  that  described 
multiple  kinds  of  equipment  or  systems  similar  in  some  key  characteristic  or 
function  in  broad,  general,  or  generic  terms. 
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3.  Interrater  Agreement.  In  terms  of  methods  of  administration  to  multiple  SMEs, 
it  appears  that  in  a  single  group,  the  better  procedure  is  to  permit  the  SMEs  to 
confer.  They  are  able  to  resolve  differences  of  task  interpretation  and  provide 
a  more  consistent  verification  and  mapping.  Furthermore,  given  the  variance 
of  the  jobs  that  comprise  a  specialty  (Drisldll  &  hfitchell,  1979),  it  should  be 
anticipated  that  there  will  be  lack  of  understanding  of  the  tasks  as  well  as  lack 
of  knowledge  of  the  WUCs  appropriate  to  any  given  tasks.  Job  incumbents  (the 
SME  raters)  simply  have  difierent  experience  which  can  limit  their  knowledge 
of  and  associated  WUCs.  Interrater  agreement  based  on  individual  SME 
ratings  should  be  expected  to  be  low.  As  far  as  reliability  of  the  verification  and 
mapping  process,  the  better  approach  would  appear  to  be  use  of  two  or  more 
groups  of  2  -  4  SMEs  as  the  basis  for  computing  agreemoit  among  groups  rather 
than  among  raters.  Further  justification  for  the  use  of  two  or  more  groups  of 
raters  are  the  differences  among  units  in  the  assignment  of  maintenance.  Job 
incumbents  at  one  base  may  not  work  on  exactly  the  same  set  of  WUCs  as  do 
the  incumbents  at  another  base. 

4.  Primary  Problems.  There  were  two  primary  problems  observed  during  the  SME 
verification  and  mapping  sessions.  First,  as  already  suggested,  was  the  problem 
of  communication— the  difficulty  that  an  SME  has  in  understanding  the  meaning 
of  tasks.  There,  obviously,  is  no  solution  to  this  problem  for  historical  data, 
except  to  permit  the  SMEs  to  cooperate  by  pooling  their  e^rience  to  arrive  at 
a  consensus  meaning.  For  future  surveys,  use  of  MDCS  data  should  improve 
communication  with  SMEs  (and  respondents)  by  providing  more  precise  (v. 
more  generic)  equipment  identification,  especially  if  weapon  system-specific  task 
lists  are  employed.  Even  these  kinds  of  tasks,  however,  cannot  be  expected  to 
eliminate,  the  communication  problem  assodated  with  the  variance  of  experience 
of  the  SMEs.  Pooling  of  SME  experience  seems  to  be  the  best  solution  for 
improving  SME  understanding  of  task  statements. 

The  second  problem  is  the  large  number  of  tasks  provided  for  SME  review  that  have 
no  possibility  of  being  matched  with  WUC  information  (52  percent  for  AFS  422X1B  and  46 
percent  for  AFS  454X3).  There  are  two  possible  solutions,  each  having  to  do  with  the  data 
screens  employed  before  the  SME  booklet  is  produced.  The  first  solution  lies  in 
improvement  of  the  WUC-AFS  crosswalk  screen.  As  will  be  discussed  later,  use  of  base 
level  CAMS  data  tapes  offer  the  most  attractive  solution. 

The  solution  to  the  second  problem  involves  reduction  of  the  number  of  tasks 
presented  to  the  SMEs  for  which  WUC  information  can  be  mapped.  This  problem  is  largely 
a  function  of  the  screen  applied  to  the  OSM  data  for  purposes  of  generating  the  job 
description  to  be  used  as  the  basis  of  the  matching.  To  provide  the  broadest  mapping  source, 
the  present  effort  selected  tasks  performed  by  15  percent  or  more  of  the  5-skill  level  workers 
who  indicated  they  worked  only  on  the  F-15.  In  future  efforts,  this  screai  can  be  adjusted 
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to  raise  the  percentage  performing  requirement  and  include  additional  screais  based  on  task 
difficulty  or  training  emphasis.  Reduction  of  the  number  of  tasks  should  not  be  significant 
from  training  and  classification  perspectives,  because  the  use  of  tasks  performed  by  small 
percentages  of  workers,  or  tasks  with  the  least  difficulty,  serve  little  purpose.  Effects  of 
changes  in  the  screens  on  the  number  and  kinds  of  tasks  to  which  SMEs  respond  should  be 
explored. 

5.  Accuracy  of  WUC-AFS  crosswalk  screening.  As  indicated  above,  the  quality 
of  the  WUC-AFS  crosswalk  was  an  inadequate  basis  for  the  WUC  screen  for 
AFS  422X1B.  A  more  accurate  crosswalk  screen  is  essential,  both  for  the 
efficiency  of  semantic  mapping  and  for  task  development.  If  anything,  the 
crosswalk  accuracy  is  more  crudal  for  developing  weapon  system-spedfic  task 
lists.  Any  WUC  omitted  during  application  of  the  SAAT  on  historical  data  can 
be  recovered  from  the  SMEs  review.  For  inventory  development,  any  missing 
WUC  equates  to  one  or  more  missing  tasks. 

6.  SME  verification.  Observation  of  the  SMEs  during  the  verification  sessions  as 
well  as  critical  review  of  the  final  mapped  product  revealed  minor  formatting 
and  algorithm  problems.  The  formatting  problem  involves  the  use  of  the  entire 
task  statement  as  the  basis  for  SME  verification  and  mapping.  The  OSM  task 
verb  (the  action  taken)  was  a  source  of  confusion.  On  several  occasions,  SMEs 
indicated  that  they  performed  maintenance  on  the  WUC  mapped  to  a  task,  but 
that  they  did  not  perform  the  action  taken  q)ecified  by  the  OSM  task  verb.  For 
example,  AFS  452X1B  personnel  indicated  they  performed  maintenance  on 
angle-of-attack  transmitters,  but  that  they  did  not  boresight  them,  as  the  OSM 
task  stated.  A  modification  that  should  eliminate  this  problem  will  be  described 
under  the  section  detailing  enhancements. 

Review  of  some  WUC-task  mapping  results  suggested  that  the  algorithms  for  mapping 
a  WUC  to  task  may  be  too  severe  or  restrictive.  There  were  instances  in  which  WUCs  were 
appropriately  mapped  to  one  task  but  not  mapped  to  another  task  having  the  same  WUC 
equipment  item.  To  preclude  these  occurrences,  the  algorithmic  parameters  for  mapping 
WUCs  to  task  statements  needs  some  minor  adjustment. 


Enhancements  of  the  SAAT 


Several  enhancements  of  the  present  SAAT  offer  the  potential  for  improving  the 
quality  of  the  OSM-WUC  data  match  provided  to  SMEs  for  verification  and  editing.  SME 
workload  can  be  minimized  by  reducing  the  tasks  they  review  to  those  that  are  performed 
in  their  AFS  and  for  which  there  is  a  high  probability  of  the  correctness  of  the  WUC 
matching.  Enhancements  also  will  reduce  the  work  associated  with  pr^aring  OSR  and  WUC 
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information  for  SAAT  processing.  The  first  three  needed  enhancements,  described  in  the 
following  paragraphs,  involve  modification  of  existing  software. 

1.  A  modification  of  the  SAAT,  that  can  be  very  important  for  improving  the 
accuracy  of  the  mapping,  involves  use  of  the  MDCS  maintenance  data  files  as 
the  data  base  for  SAAT  mapping.  There  are  two  improvements  that  may  result 
from  use  of  the  maintenance  data  base:  (a)  informational  value  of  tokens  used 
for  mapping  can  be  sharpened;  and  (b)  the  array  of  WUCs  used  in  mapping  will 
be  limited,  thus  reducing  the  number  of  extraneous  WUCs  appearing  in  the  SME 
booklet. 

First,  present  SAAT  matching  is  based  on  the  "information  value"  of  tokens  (words, 
phrases,  concepts)  that,  simply  stated,  is  calculated  from  the  frequency  of  occurrence  of  the 
tokens’  occurrence  within  their  data  set  as  compared  to  probability  of  occurrence  outside  the 
data  set.  A  given  token  in  the  WUC  data  set,  for  example,  has  a  value  based  on  its 
frequency  of  occurrence  within  that  set.  The  same  is  true  for  any  token  in  the  OSM  data  set. 

There  are  data,  however,  contained  in  the  MDCS  and  OSM  data  bases,  associated 
with  the  tokens  and  which  have  merit  for  enhancing  the  "information  value"  of  the  tokens 
with  which  the  data  are  associated.  For  the  MDCS,  the  frequ^cy  of  occurrence  of 
maintenance  actions  on  a  particular  WUC  token  can  be  used  in  computing  the  information 
value  of  the  token.  Percent  performing  and  percent  time  spent  data  are  tey  data  elements 
for  weighting  the  information  value  of  OSM  tokens.  Obviously,  within  the  data  set  and 
outside  the  data  set  probability  of  occurrence  values  would  be  affected  by  use  of  the 
additional  data.  The  result  of  these  effects  would  be  an  improvement  of  the  overall  mapping. 

This  modification  of  the  SAAT  involves  changes  in  both  the  mapping  algorithms  and 
microcomputer  programs  to  permit  accessing  the  WUCs  and  OSM  task  data  essential  for 
computing  the  weighting  values.  The  modification  also  involves  changing  the  WUC  data  set 
that  is  presently  a  basis  of  the  mapping  process.  This  change  involves  another  needed 
enhancement  discussed  in  the  following  paragraph. 

Use  of  the  MDCS  maintenance  data  files  would  also  provide  a  source  for  two  further 
screens  (in  addition  to  the  WUC-AFS  screen),  applied  to  WUCs  before  mapping,  by  limiting 
the  data  base  used  for  SAAT  mapping  to  only  those  WUCs  that  are  reported.  First,  as  the 
SAAT  was  originally  designed,  the  B-4  Master  File  is  the  WUC  data  set  used  for  mapping 
to  OSM  tasks.  The  B-4  Master  file  contains  all  of  the  work  unit  codes  pertaining  to  a  given 
aircraft,  communication-electronics  system,  aircraft  support  equipment  (such  as  Aerospace 
Ground  Equipment),  or  missile.  When  the  fifth  character  level  of  WUC  is  included,  a  vay 
specific  level  of  definition  exists,  and  the  number  of  codes  normally  runs  into  the  thousands. 
As  a  result  there  is  a  high  probability  of  matching  a  token  from  an  inappropriate  WUC  to 
an  OSM  task.  Further,  at  least  in  many  AFSs,  job  incumbents  do  not  perform  maintenance 
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at  this  level.  The  SAAT  mapping  is  therefore  not  as  accurate  and  representative  of  an  AFS 
as  it  could  be. 

By  changing  the  data  set  from  the  B-4  Master  File  to  that  obtained  from  the 
maintenance  data  tape  for  the  aircraft  or  other  interest  item,  only  the  work  unit  codes  that 
personnel  report  for  work  on  the  aircraft  of  interest  will  comprise  the  data  set  used  for 
mapping.  This  data  set  will  represent  the  complete  set  of  WUCs.  Extraneous  WUCs,  either 
because  the  WUCs  are  inappropriate  to  the  AFS  of  interest  or  at  a  too  specific  WUC  level, 
would  not  be  available  for  inappropriate  mapping. 

Second,  these  WUCs  could  be  further  screened,  since  it  is  possible  that  some  WUCs 
that  have  low  frequencies  of  repair  should  be  screened  before  mapping,  especially  if  task  list 
length  is  at  issue.  These  are  the  WUCs  for  which  there  is  the  greatest  probability  that  the 
Job  Invaitory  for  the  AFSs  will  not  have  an  appropriate  task  or  very  few  job  incumbents 
report  that  they  perform. 

2.  A  second  modification  involves  the  use  of  a  more  strict  screen  in  conjunction 
with  one  or  more  add-back  features  applied  to  OSM  tasks.  This  modification 
would  result  in  a  reduced  SME  workload  by  providing  tasks  that  are  most  likely 
to  be  performed  by  the  AFS  on  the  weapon  system,  or  other  equipment,  of 
interest.  As  the  SAAT  has  been  applied  to  date,  OSM  tasks  have  had  two 
screens  applied.  In  previous  testing,  the  screens  limit  the  tasks  to  those 
performed  by  15  percent  or  more  job  incumbents  who  work  only  of  the  F-15. 
As  the  results  of  the  verification  and  editing  of  mapped  data  by  SMEs  with  AFS 
452X1B  and  454X3  indicated,  a  large  percentage  of  tasks  (26  and  29  percent, 
respectively)  were  not,  according  to  the  SMEs,  tasks  that  incumbents  of  the  AFS 
should  perform,  or  were  not  tasks  that  could  be  performed  on  the  F-15.  Review 
of  the  tasks  indicates  that  most  of  them  are  performed  by  small  percentages  of 
personnel. 

Investigation  of  changes  in  the  screens  as  weU  as  the  use  of  add-back  options  is 
needed  to  eliminate,  or  at  least  reduce,  the  number  of  spurious  tasks.  One  possible  change 
involves  setting  the  minimum  percent  performing  screen  at  the  mean  (or  some  such  standard) 
percentage  of  incumbents  performing  tasks.  Potential  add-back  options  include  use  of  task 
difficulty  and  training  emphasis  ratings.  These  ratings  could  be  applied  to  add  any  task  for 
which  probable  relevance  to  the  AFS  is  high,  such  as  one  standard  deviation  above  mean 
difficulty  rating  and  above  mean  training  emphasis  rating.  Use  of  the  difficulty  oiterion 
would  ensure  that  relevant  tasks  performed  by  percentages  not  meeting  the  minimum 
percentage  cutoff  (as  is  ftequently  the  case  for  difficult  tasks)  would  be  included  in  the 
mapping.  Training  emphasis  ratings  in  a  sense  are  an  indication  of  the  relevance  of  a  task 
to  the  work  of  a  specialty.  Thus,  any  task  exceeding  the  mean  value  probably  should  be 
added  even  though  it  did  not  meet  the  minimum  percentage  performing  cutoff. 
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3.  Another  modification  involves  the  formatting  of  the  SME  booklet.  As  discussed 
in  preceding  paragraphs,  SMEs  identified  tasks  containing  WUCs  that  they 
reported  but  not  for  the  action  taken  rq)resented  by  the  OSM  task  statement. 
Hindsight  suggests  that  the  SME  booldet  should  not  include  the  entire  task 
statement,  because  some  of  the  SMEs’  confusion  during  the  verification  and 
editing  process  c^itered  on  the  question  of  their  performance  of  the  action  taksn 
implied  by  the  OSM  verb.  The  SAAT  OSM  task-WUC  mapping,  that  the  SME 
uses  for  verification  and  editing,  is  based  on  the  use  of  the  task-verb  objects, 
essentially  the  noun  phrases,  in  the  OSM  task  statements.  It  is  not  until  after 
SME  verification  that  task  verbs  and  actions  taken  are  mapped.  Mapping  against 
the  objects  would  not  detract  from  the  quality  of  the  final  data,  because  the 
maintenance  data  from  the  MDCS  data  files,  showing  the  action  taten  on  the 
WUCs,  will  indicate  whether  job  incumbents  perform  the  action  taken  implied 
by  the  OSM  task  verb.  Present  SAAT  software  should  be  modified  to  change 
the  SME  booklet  format  to  one  in  which  WUCs  are  mapped  to  the  task  verb 
objects  or  noun  phrases. 

4.  The  fourth  modification  involves  the  testing  of  various  parameters,  such  as  the 
within-set  and  outside-set  values,  for  matching  WUCs  to  OSM  tasks.  Minor 
software  revision  would  be  required  to  permit  variation  of  the  levels  of  the 
parameters.  At  least  some  of  the  testing  and  validation  of  the  within-  and 
outside-set  parameters  could  be  accomplished  on  OSM-MDCS  mapping  data 
obtained  from  previous  tryouts. 

The  remaining  enhancements  are  needed  to  improve  the  quality  of  the  final  mapped 
product,  and  minimize  the  time  and  work  of  SAAT  preprocessing  and  SME  verification  and 
editing.  They  relate  to  the  acquisition  of  better  sources  of  WUC-AFS  crosswalk  information 
and  an  automated  source  of  fully-expanded  WUC  titles.  These  enhancements  are  described 
below. 

1.  A  better  source  of  WUC-AFS  orosswalk  data  is  critically  needed,  since  the 
LCOM  Percent  Contribution  Reports  used  in  previous  SAAT  research,  although 
readily  accessible,  are  inadequate.  So  fer,  three  approaches  to  obtaining  WUC- 
AFS  crosswalk  information  have  been  used.  First,  through  personal  interviews, 
SMEs  used  a  WUC  manual  to  check  the  WUCs  they  reported  when  they 
performed  maintenance.  This  approach  was  highly  effective,  but  more  resource 
intensive  than  is  desirable  (except  as  a  last  resort).  Second,  the  LCOM  Percent 
Contribution  Report  has  been  employed,  and  the  results  of  the  F-15  mapping 
research  indicates  it  is  inadequate  for  mapping.  For  use  in  developing  weapon 
system-specdfic  tasks,  it  is  totally  inadequate,  because  the  WUCs  employed  in 
LCOM  work  are  usually  truncated  to  the  3-<hgit  level. 
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Third,  a  mail  survey  has  been  employed.  Technicians  with  AFS  452X3A,B,C  were 
mailed  a  booklet  in  which  possible  WUCs  which  they  would  report  when  working  on  the  B- 
IB  were  included.  The  WUCs  provided  in  the  booklet  were  extracted  from  the  preliminary 
WUC  for  the  B-IB.  The  technicians  were  asked  to  check  the  WUCs  they  reported  and 
write-in  any  WUCs  that  had  changed  or  was  not  included  in  the  list  provid^.  Nine  of  ten 
SMEs  (one  SME  had  been  reassigned)  complied  with  the  request.  Numerous  WUCs  were 
written-in.  Most  write-ins  were  the  result  of  amendments  to  the  manual  from  which  the 
WUC  extracts  used  in  the  booklet  provided  to  the  SMEs  were  made.  The  primary  problem 
noted  was  that  some  of  the  SMEs  did  not  provide  the  WUCs  in  the  amendments  but,  instead, 
only  referred  to  the  amendment  number  for  the  WUCs.  Fortunately,  for  this  effort,  other 
SMEs  provided  the  spedfic  WUCs  included  in  the  amendments  which  were  not  available  to 
the  project  team.  Any  future  use  of  a  survey  methodology  should  incorporate  explicit 
instructions  to  transcribe  all  WUCs  rather  than  refer  to  manual  amendments.  The  quality 
of  the  survey  responses  were  not  verified,  since  the  data  were  used  for  producing  a  B-IB- 
spedfic  task  list,  not  for  mapping  (in  the  mapping  process,  SMEs  would  be  verifying 
accuracy  of  WUCs  along  with  mapping  the  codes  to  OSM  task  statements).  Thus,  there  was 
no  opportunity  to  obtain  further  SME  verification.  It  is  clear,  however,  that  in  some  cases 
the  SMEs,  checked  different  WUCs.  This  fact  is  not  une}q)ected.  Not  all  incumbents  of  an 
AFS  perform  the  same  work;  any  given  SME  may  not  be  familiar  with  aU  of  the  WUCs  for 
the  AFS. 

While  there  are  the  three  other  potential  sources  of  WUC-AFS  crosswalk  information 
just  described,  the  best  source  can  be  found  in  base  level  CAMS  data  tapes.  As  indicated 
in  AIR  FORCE  CROSSWALK  (1989),  in  the  approximately  60  percent  of  units  employing 
CAMS,  WUC-AFS  data  are  stored  on  base-level  data  tapes  for  a  period  of  up  to  six  months. 
Procedures  should  be  established  for  obtaining  an  extract  of  these  tapes  showing  the  WUC- 
AFS  crosswalk  from  at  least  two  bases  maintaining  the  aircraft  for  which  mailing  is  desired. 
Minor  software  development  would  be  required  for  preprocessing  the  data  for  SAAT 
processing. 

Since  WUC-AFS  crosswalk  information  is  cracial  for  future  SAAT  applications  or 
inventory  development  efforts,  access  to  CAMS  base  level  data  tapes  is  very  important. 

2.  By  far  the  most  time  consuming  and  confusing  preprocessing  is  the  e^qyansion 
of  the  abbreviated  WUC  titles,  which  are  the  basis  of  the  mapping  process.  It 
is  one  of  the  most  crucial  stq)s  required  for  applying  the  SAAT.  The  B-4 
Master  File  allocates  only  19  characters  for  the  WUC  title.  Therefore, 
abbreviations  must  be  used  which  are  not  only  short  but  are  inconsistent  from 
one  usage  to  another.  Each  abbreviation  must  be  expanded  to  the  full  word  for 
which  it  stands.  To  date,  no  source,  automated  or  otherwise,  has  been  available 
for  the  process. 
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At  this  time,  the  more  promising  of  two  potential  solutions  is  the  ca'eation  of  a  kind 
of  thesaurus  or  look-up  table  based  on  the  expansions  of  the  WUC  titles  for  the  F-15  and  F- 
16  already  accomplished.  In  accomplishing  these  expansions,  a  very  large  number  of 
duplications  of  WUC  abbreviations  between  the  two  aircraft  were  observed.  Similar 
duplications  for  other  aircraft  can  be  anticipated-most  aircnaft,  for  example,  will  have  fuel 
quantity  indicators,  speed  indicators,  altimeters,  transmitters,  and  so  on.  While  there  are 
variations  of  the  abbreviations  that  may  be  used  for  these  items,  once  the  abbreviations  are 
identified  and  expanded,  they  readily  translate  to  other  WUC  titles  for  other  aircraft  or 
systems. 

It  is  likely  that  a  look-up  table  containing  most  of  the  abbreviations  and  their  expanded 
forms  can  be  constructed  firom  the  F-15  and  F-16  ejq)ansions.  Arrangement  of  the  WUC  in 
the  look-up  table  should  be  based  on  some  structure  like  that  to  be  found  in  DoD  Standard 
863B  or  in  MIL-M-38769C  (USAF).  This  structure  would  assist  in  identifying  the  preferred 
expansion  of  an  abbreviation  for  which  there  are  two  or  more  options.  Subsequently,  the 
look-up  table  would  be  amended  as  new  abbreviations  were  encountered.  This  look-up  table 
could  be  incorporated  in  the  SAAT  software  so  that,  during  expansion  of  WUC  titles,  the 
first  step  would  be  a  look-up  of  an  abbreviation  when  it  occurs. 

Very  late  in  the  course  of  this  project,  information  was  received  from  the  Air  Force 
logistics  community  that  a  soxurce  may  exist  in  which  the  fully-expanded  titles  are  available 
in  electronic  format  and  on  microfiche.  According  to  the  information,  part  number,  WUC, 
and  the  full  titles  are  available  in  the  Mission  Incapable  Report  (MICAP).  This  source 
should  be  investigated  further. 

The  SAAT  is  friendly  software,  but  it  is  not  menu  driven  in  the  sense  that  aft 
explanatory  information  is  not  displayed  on  the  screen.  The  documentation,  however,  is  very 
explanatory.  Any  computer  technician  familiar  with  microcomputers  and  with  CODAP  setup 
and  use  of  the  AFHRL  UNISYS  1100/82  can  use  the  SAAT. 


V.  FACTORS  BEARING  ON  THE  DEVELOPMENT  OF  WEAPONS 

SYSTEM-SPECIFIC  TASKS 


The  basis  for  developing  weapon  systems-specific  tasks  to  be  used  in  the  OSM  survey 
methodology,  rests  in  the  use  of  microcomputer  methods  for  extracting  records  of  the  actual 
maintenance  performed  on  aircraft  during  a  given  period  of  time  from  MDCS  files.  Job 
Inventory  tasks,  used  in  future  OSM  surveys  based  on  these  maintenance  records,  or  at  least 
precoding  of  OSM  tasks  with  MDCS  work  unit  and  action  taken  codes,  can  provide  highly 
specific  aircraft  data.  They  also  offer  the  potential  for  reducing  or  eliminating  the  need  for 
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Semantic-assisted  Analysis  Technology  mapping  of  OSM  and  MDCS  historical  task  level 
data.  Development  of  such  methods  not  only  can  provide  the  specific  level  of  information 
needed  for  certain  manpower,  personnel,  and  training  purposes,  but  they  offer  the  potential 
for  improving  the  coverage  and  quality  of  OSM  maintraance  task  lists. 

This  section  will  address  the  use  of  maintenance  data  to  develop  specific  tasks  firom 
several  perspectives  which  follow  the  requirements  for  developing  tasks  that  elicit  reliable 
job  incumbent  data,  the  basis  of  current  OSM  inventory  development  and  data  collection 
practices,  the  levels  of  task  description  that  facilitate  the  various  uses  of  OSM  data,  and  the 
scope  of  the  need  to  develop  weapon  system-specific  task  lists. 


Task  Characteristics 

There  are  five  characteristics  that  drive  the  development  of  tasks  included  in  a  OSM 
Job  Inventory  (Morsh  &  Archer,  1967;  Driskill  et  al.,  1987).  First,  tasks  should  be 
expressed  in  language  that  communicates  clearly  to  the  respondents;  second,  tasks  should  be 
mutually  exclusive  or  independent  of  one  another;  third,  tasks  should  be  expressed  at  about 
the  same  level  of  specificity;  fourth,  they  should  differentiate  among  workers;  and  fifth,  they 
should  be  time  ratable.  The  first  four  characteristics  have  implications  for  the  approach  used 
in  building  specific  task  inventories  from  maintenance  data. 


Communication 

MDCS  data  to  develop  OSM  tasks  can  be  most  useful  in  preparing  tasks  that 
communicate  clearly  with  respondents.  In  the  mapping  of  F-16  and  F-15  data,  there  were 
instances  noted  where  the  generic  nature  of  the  tasks  led  to  misunderstanding  (Driskill  et  al., 
1987).  For  mapping  purposes,  the  OSM  job  description  employed  was  for  5-skill  level 
persoimel  worldng  only  on  the  F-16.  Nevertheless  according  to  F-16  subject  matter  experts 
reviewing  the  OSM-MDCS  mapping,  more  that  20  percent  of  these  persoimel  indicated  they 
perform^  tasks  on  systems  that  are  not  on  any  model  of  F-16.  Wagner  (1986)  also  not«i 
the  same  problem.  The  testing  of  SAAT  on  F-15  data  in  this  study  also  revealed  this 
problem.  MDCS  data  can  lead  to  a  more  comprehensible  e^qrression  of  systems,  subsystem, 
and  components. 


Independence 

In  terms  of  the  independence  of  tasks,  use  of  MDCS  data  in  developing  task 
statements  has  two  sides.  The  lack  of  independence  of  tasks  noted  in  present  OSM  practice 
(see  Wagner,  1986,  for  example)  can  be  avoided  if  MDCS  data  is  used  in  task  development. 
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Task  statements  appearing  in  inventories  generally  reflect  work  at  the  third  character  (system) 
of  the  MDCS  data;  however,  most  inventories  contain  some  task  items  that  describe  work 
at  the  fourth  and,  sometimes,  the  fifth  character  of  the  same  systems.  Redundancy  occurs 
when  tasks  relating  to  the  same  item  of  equipment  are  developed  at  two  or  more  of  these 
levels.  Use  of  the  MDCS  during  OSM  task  development  should  preclude  this  kind  of 
redimdancy  and  enhance  task  indq>endence. 

The  other  side  of  using  MDCS  data,  however,  is  that  the  same  or  highly  similar  WUC 
equipment  descriptors  may  be  used  for  several  different  aircraft.  For  example,  the  Job 
Inventory  for  AFS  423X3  has  this  task:  Remove  or  install  hydraulic  radiators  or  fuel  oil  heat 
exchangers.  This  task  is  generically  stated  and  is  intended  to  represent  work  performed  on 
fuel  systems  of  numerous  aircraft.  On  at  least  two  of  these  aircraft,  the  F-16  and  the  F-15, 
the  WUC  descriptor  is  the  same:  Fuel  oil  heat  exchanger  (WUC  46ADE  for  the  F-15  and 
WUC  46APO  for  the  F-16).  If  these  descriptors  were  used  separately  (as  in  separate  task 
lists  for  each  aircraft)  to  desCTibe  work  performed  on  the  two  aircraft,  each  task  statement 
must  be  modified  by  the  aircraft  designator  (e.g.  F-15  fuel  oil  heat  exchanger).  Otherwise, 
in  a  consolidated  job  description  across  airCTaft  for  the  AFS,  both  tasks  would  appear  to  be 
identical,  when  in  feet  they  are  different  for  the  F-15  and  F-16.  Develqpmait  of  weapon 
system-specific  tasks  must  take  into  account  WUC  descriptor  similarities  if  the  data  are  to 
be  accurately  aggregated  across  aircraft  systems  for  different  aircraft  types. 


Specificity 

Since  tasks  in  inventories  describe  work  at  different  WUC  levels,  task  specificity 
varies.  Sometimes,  work  performed  on  one  system  is  described  at  the  system  or  subsystem 
level  (third  and  fourth  character).  Other  times,  work  performed  on  a  system  will  be 
described  at  the  component  level  (fifth  character).  Differences  of  specificity  impact  the 
computation  of  percentage  of  time  spent  on  task  performance.  More  percent  time  spent  will 
be  computed  for  systems  that  are  more  specifically  described.  Use  of  MDCS  data  to  develop 
task-  lists  can  reduce  differences  of  specificity  by  standardizing  the  WUC  level  at  which  task 
lists  are  developed. 


Differentiation 

It  is  important  to  have  differentiation  among  workers  (for  example,  identify 
differences  in  the  tasks  performed  by  ejq)erienced  and  inexperienced  job  incumbents).  The 
MDCS  data  are  based  on  a  fixed  set  of  action  taken  codes.  That  may  be  inadequate  for 
providing  the  discrimination  among  worker  experience  required  for  OSM  tasks.  OSM  task 
development  does  not  use  a  fixed  set  of  verbs  (or  actions  taken).  Instead,  the  verbs,  elicited 
from  subject  matter  experts,  are  used  to  provide  for  differentiation  among  workers  in  the 
AFS. 
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In  the  SAAT  mapping  of  OSM  to  MDCS  tasks,  OSM  task  verbs  were  converted,  with 
the  cooperation  of  USAFOMC  inventory  development  personnel  to  MDCS  action  taken  codes 
(Driskill  et  al.,  1987)  prior  to  SAAT  mapping  for  purposes  of  matching.  The  reverse, 
working  from  MDCS  tasks  to  OSM  tasks,  can  be  followed  for  developing  task  statements. 
To  provide  for  more  accurate  OSM  task  description  during  inventory  development,  a 
glossary  of  verbs  should  be  compiled.  Such  a  tool  would  show  the  relationship  of  OSM 
verbs  to  corresponding  MDCS  action  taken  codes  with  guidelines  prescribing  the 
differentiating  qualities  and  how  they  should  be  applied.  In  addition,  specification  of  the 
method  of  matching  task  verbs  to  actions  taken  codes  will  be  required. 

In  addition  to  these  task  characteristics,  there  is  an  implicit  requirement  that  the  tasks 
extensively  cover  the  work  of  chosen  occupations  being  described.  While  it  is 
unquestionably  true  that  any  task  list  is  only  a  sample  of  the  work  performed,  that  sample 
of  tasks  should  represent  a  comprehensive  description  of  the  various  types  of  work  in  the 
various  jobs  making  up  an  occupation.  MDCS  data  should  provide  this  comprehensive 
coverage.  As  pointed  out  in  an  earlier  report  (Driskill  et  al.,  1987),  various  sources  have 
questioned  the  reliability  of  these  data.  For  purposes  of  developing  task  lists,  however, 
reliability  of  worldoad  data  is  not  an  issue.  The  goal  is  to  generate  a  listing  of  the  tasks 
perform^  in  the  maintenance  of  a  given  aircraft  or  piece  of  equipm^t.  The  MDCS  data 
base  of  maintenance  performed  across  the  force- wide  inventory  of  a  specific  aircraft  or  piece 
of  equipment,  should  provide  comprehensive  coverage  of  all  WUC  equipment  items  and 
actions  taken  on  these  items. 


Carrent  Liventorv  Development  and  Data  Collection  Policies 

Implications  of  the  change  in  the  nature  of  tasks  used  to  collect  occupational  survey 
data  are  discussed  in  this  section.  Two  perspectives  are  reviewed.  First,  current  inventory 
development  practices  are  described,  and  the  rationale  for  these  practices  are  givra.  Second, 
implications  of  changes  in  the  character  of  Job  Inventories  on  present  data  collection  and 
analysis  procedures  are  enumerated.  Information  for  these  discussions  were  obtained  from 
two  sources:  a  large  number  of  Job  Inventories  used  in  OSM  surveys  of  maintenance 
specialties  during  the  past  9  years,  and  interviews  of  USAFOMC  personnel. 


friventorv  Developinent  Practices 

Review  of  the  job  inventories  led  to  four  conclusions: 

o  Job  Inventories  include  all  of  the  tasks  describing  the  work  comprising  a 
specialty  or  specialties  being  surveyed. 
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o  Job  Inventory  includes  general  and  generic  tasks  such  as  those  pertaining 

to  management,  supervision,  and  training. 

o  As  the  number  of  weapon  systems  covered  by  an  AFS  increases,  the  generality 
and  number  of  generic  tasks  increases  due  to  the  desire  to  reduce  task  list 
length. 

0  OSM  task  statements  consistently  begin  with  verbs  or  verb  phrases,  but  the 
semantics  of  the  verbs  or  verb  phrases  are  inconsistent. 

The  following  paragraphs  provide  further  e:^lanation  of  each  of  these  conclusions. 

In  regard  to  Job  Inventory  coverage,  excq)t  in  very  rare  cases,  tasks  describing  work 
performed  by  personnel  with  specialty  shreds  are  included.  Also,  several  specialties  and 
their  associated  shreds  may  be  combined  in  a  single  inventory.  This  practice  is  dictated  by 
the  need  for  determining  the  similarity  of  task  performance  among  the  shreds  or  specialties. 
In  defining  similarity  of  task  performance,  survey  respondents  must  have  the  same  frame  of 
task  reference  for  their  time  spent,  task  difficulty,  training  emphasis,  or  other  factor  ratinp. 
If  the  task  data  are  not  collected  in  a  single  inventory  some  method  for  benchmarking 
tasks  among  inventories  is  required  to  equate  ratings  across  tasks. 

Second,  each  Job  Inventory,  describing  either  single  or  multiple  systems,  includes 
genCTal  and  generic  tasks  (excluding  tasks  comprising  Job  Inventory  Duties  A,  B,  C,  and  D, 
such  as  those  pertaining  to  management,  supervision,  and  training).  These  tasks  describe 
work  job  inrnmbpnts  perform  in  their  specialty  or  specialties.  Each  of  these  two  categories 
consist  of  two  classes  of  tasks;  they  are  differentiated  because  they  serve  different  purposes 
in  inventories. 

Aircraft-eonipment  general  tasks  are  used  to  describe  work  performed  across  aU 
aircraft  and  associated  systems  by  job  incumbents  without  regard  to  AFS  or  equipment  items 
whose  WUCs  are  not  included  under  the  aircraft  WUCs.  The  WUCs  for  the  latter  class  of 
items  are  provided  in  separate  publications  and  their  maintenance  is  usually  the  responsibility 
of  other  specialties.  Examples  of  some  aircraft-equipment  general  tasks  are  as  follows:  ^gk 
aircraft.  Fuel  and  defuel  aircraft,  ralihrate  simulators  or  mockups.  Clean  test  station  filters. 
Position  maintenance  stands,  and  Position  powered  or  nonpowered  AGE  to  aircraft. 

Under  the  MDCS,  work  supporting  a  specific  aircraft  not  related  to  specific  aircraft 
system  maintenance,  or  which  is  related  to  general  kinds  of  maintenance  (i.e.,  corrosion 
control)  is  normally  recorded  under  Support  General  or  Inspection  WUCs.  General  task 
statements  usually  are  employed  to  describe  work  performed  under  these  codes  as  well  as 
to  describe  work  on  equipment  for  which  the  specialty  does  not  have  primary  responsibility. 
For  equipment  which  is  not  a  part  of  a  basic  aircraft  system,  work  performed  by  incumbents 
of  a  specialty  with  primary  maintenance  responsibility  on  that  equipment  is  recorded  using 
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Work  Unit  Code  manuals  for  the  specific  equipment.  The  WUCs  and  ATCs  in  these 
manuals  can  be  used  to  develop  tasks  for  the  specific  equipment.  Use  of  the  Support  General 
and  Inspection  Codes  (recordhig  production  credit  of  repetitive  tasks  of  a  general  nature)  to 
develop  general  tasks  applicable  across  aircraft  and  specialties  should  be  e^lored,  since  there 
is  a  high  degree  of  similarity  among  these  codes  and  task  statements  across  aircraft.  These 
codes  appear  in  each  WUC  manual  and  are  standard  across  manuals. 

APS  general  tasks  are  used  to  describe  work  that  cannot  be  directly  related  to  a  WUC 
descriptor  and,  in  most  cases,  is  performed  almost  exclusively  by  members  of  a  specific 
APS.  These  tasks  desoibe  work  required  to  support  the  maintenance  of  specific  aircraft 
systems,  listed  in  WUC  manuals.  Normally,  work  involving  these  support  maintenance  tasks 
is  recorded  in  the  MDCS  imder  Support  General  Codes.  They  are  necessary  to  account  for 
the  full  range  of  specialty  incumbents’  performance.  The  following  examples  were  extracted 
from  the  Puels  Specialty  to  illustrate  APS  general  tasks;  Direct  hydrazine  spill  clean-up 
procedures:  Notify  fire  department  of  fuel  systems  maintenance,  and  Position  drip  pans. 
SME  input  for  the  development  of  Support  General  tasks  is  necessary. 

Two  classes  of  generic  tasks  are  WUC  nomenclature-generic  tasks,  and  methods, 
procedures,  or  materials  generic  tasks.  WUC  nomenclature-generic  tasks  are  used  to 
describe  aircraft  system  maintenance  or  off-aircraft  equipment  maintenance  tasks  when  there 
are  similarities  of  the  tasks  across  multiple  airaaft,  or  similaiities  of  off-aircraft  equipment 
items,  maintained  by  a  single  specialty.  The  WUC  nomenclature  from  which  the  generic 
tasks  are  generated  can  be  found  in  the  individual  aircraft  or  equipment  WUC  manuals. 
Under  present  technology,  use  of  generic  tasks  cannot  be  avoided.  They  are  utilized  to 
describe  systems  that  are  similar  in  varying  degrees  across  several  aircraft  or  aircraft  systems 
(e.g.,  engine  supply  systems,  fuels,  instruments,  flight  controls).  A  task  list,  for  a  specialty 
covering  multiple  aircraft  or  equipment  items,  could  quicldy  become  too  long  for  mail  survey 
administration  under  existing  procedures,  if  sq)arate  tasls  were  written  for  each  piece  of 
equipment  on  each  aircraft. 

The  second  class  of  materials  generic  tasks  describe  methods,  procedures,  or  materials 
that  may  be  used  to  repair  or  maintain  systems,  subsystems,  or  components  maintained  by 
a  specMty.  Typically,  since  the  tasks  can  be  performed  on  a  variety  of  systems,  no 
reference  is  made  to  any  WUC  nomenclature.  Work  described  by  these  tasks  is  peculiar  to 
a  given  specialty  and  may  be  accounted  for  under  MDCS  ATCs  and  WUCs.  Typical  tasks 
of  this  type  are  found  in  specialties  like  AFS  427X5,  Airframe  Repair  Specialist,  and  AFS 
427X1,  Corrosion  Control  Specialist. 

The  third  conclusion  is  that  not  only  do  ways  used  to  describe  tasks  vary,  but  as  the 
number  of  weapon  systems  covered  by  an  AFS  increases,  the  number  and  generality  of  the 
descriptors  for  the  generic  tasks  increase  as  well.  Primarily  these  increases  result  because 
of  the  need  to  reduce  task  list  length.  In  single  aircraft  AFS  like  those  for  the  F-111,  F-15, 
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F-16,  and  the  B-IB,  task  list  length  is  not  at  issue,  and  many  of  the  tasks  referring  to  specific 
WUC  equipment  nomenclature  are  written  at  the  fourth  character  (subsystem)  level  of  the 
WUC.  Production  of  these  tasks  from  MDCS  data,  using  microcomputer  methodology 
already  developed,  can  be  accomplished.  Further,  even  if  the  microcomputer  methods  for 
deriving  task  statements  from  MDCS  data  are  not  used,  after-the-fact  SAAT  mapping  is 
straight  forward  because  the  descriptive  terms  used  in  OSM  tasks  are  generally  similar  to 
MDCS  equipment  nomenclature. 

In  inventories  covering  a  few  aircraft  or  equipment  items  or  an  AFS  that  has  major 
command  shreds,  as  in  AFS  455X2,  Communication  and  Navigation  Systems  Specialist,  tasks 
are  reasonably  identifiable  with  WUC  nomaiclature.  Although  generic,  they  are  usually 
written  at  the  third  and  fourth  (subsystem  and  sub-subsystem)  levels  of  the  WUC.  For 
example,  here  are  two  such  tasks:  Isolate  malfunctions  to  compass  amplifiers,  and  Rempy^ 
or  replace  rtyonnaissance  adapter  units.  For  such  specialties,  the  length  of  the  task  list  is 
unlikely  to  become  too  long.  SAAT  mapping  is  more  effective  for  these  AFS  than  for  those 
AFSs  which  cover  maintenance  of  all  aircraft. 

Tasks  for  specialties  covering  all  or  many  aircraft  or  equipment  take  on  two  different 
generic  forms.  For  six  specialties  lilffi  AFS  427X5,  Airframe  Rq)air  Specialist,  and  AFS 
427X1,  Corrosion  Control  Specialist,  aircraft  system  references  are  not  used.  Methods, 
procedures,  or  materials  generic  tasks  are  primarily  utilized,  such  as  these  two  examples 
from  the  Job  Inventory  for  AFS  427X5:  Perform  solid  laminate  repairs,  and  Cut 
replacement  honeycomb  core  using  cylindrical  core  cutter.  It  is  q^uestionable  whether  weapon 
system-specific  tasks  should  be  developed  for  the  few  specialties  like  these.  Also,  as  the 
tasks  are  presently  described,  semantic-assisted  analysis  techniques  cannot  be  applied. 

In  specialties  like  Airframe  Repair  and  Corrosion  Control,  the  methods,  procedures, 
or  matf^riais  generic  tasks  are  more  efficient  and  descriptively  more  parsimonious.  From 
personnel  classification,  promotion  testing,  and  training  perspectives,  the  kinds  of  airframe 
parts  upon  which  the  specialist  works  are  not  nearly  so  important  as  what  is  done  to  repair 
or  m?»intain  the  material  from  which  the  parts  are  made.  If  tasks  were  written  around  WUC 
identification,  either  the  type  of  material  from  which  airframe  parts  are  made  would  have  to 
be  derived  from  analysis  subsequent  to  the  survey;  or  a  separate  task  for  each  part  made  of 
a  different  material  with  an  indication  of  the  type  of  material  would  have  to  be  written.  It 
would  appear  to  be  very  difficult  if  not  impossible  to  develop  generic  task  statement  based 
upon  equipment  nomenclature  and  provide  information  about  materials. 

Tn  a  larger  number  of  specialties  like  AFS  454X3,  Fuels  System  Mechamc,  tasks  that 
generally  target  aircraft  system  and  components  are  included  as  well  as  tasks  that  are  similar 
in  character  to  those  in  the  Airframe  specialty,  such  as  these:  Perform  red  talcum  powder 
tests:  Apply  protective  topcoat  sealants.  Design  of  an  intelligent  system  for  developing  the^ 
kinds  of  tasks  is  made  very  difficult.  Furthermore,  semantic-assisted  analysis  techniques  will 
not  map  these  tasks  with  WUC  data.  The  mixing  of  the  generic  tasks  describing  work 
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processes,  and  those  targeted  at  WUC  nomenclature  tasks  in  a  single  inventory  also  provide 
the  setting  for  overlapping  or  redundant  tasks. 

When  these  broad  specialties  contain  WUC  nomenclature-generic  tasks  (that  is,  tasks 
that  target  similar  systems  across  aircraft),  task  content  may  r^resent  WUCs  third,  fourth, 
fifth  character  (sub-subsystem),  or  a  combination  of  these  levels.  No  evidence  has  been 
identified  that  suggests  a  systematic  approach  to  identify  the  levels  to  be  used.  Examples  of 
WUC  nomenclature-generic  tasks  from  the  Job  Inventory  for  AFS  454X3  are:  Isolate 
malfnncitinns  of  air  refiieling  systems  of  receiver  aircraft:  Remove  or  install  air  refueling 
recq)tacles. 

The  last  two  examples,  although  they  refer  to  descriptors  used  in  WUC  manuals,  are 
in  a  sense  "aggregated"  tasks;  they  are  intended  to  describe  the  work  performed  on  any 
aircraft  having  air  refueling  receiver  systems.  Generally,  they  are  written  at  the  third 
character  (subsystem)  level  of  the  WUC  for  aircraft.  In  the  example  tasks  above,  selection 
of  a  generic  desoiptor  was  suggested  by  the  commonality  of  equipment  nomenclature  for  air 
refueling  systems  across  aircraft.  Sometimes,  however,  a  generic  descriptor  must  be  created 
to  classify  the  aggregation  of  equipment  to  be  described,  because  of  equipment  nomenclature 
variations  across  aircraft  WUCs. 

To  further  illustrate  this  point,  the  Fuels  Specialty  has  numerous  valves  with  WUCs 
for  various  aircraft.  Although  frequently  WUC  descriptors  will  not  be  included  to  indicate 
whether  a  valve  is  a  "check"  valve  or  a  "float"  valve,  task  statements  will  be  written  like 
these:  Ttcmove  or  replace  check  valves,  and  Remove  or  replace  float  valves.  The 
assumption  is  made  that  all  job  incumbents  will  know  which  valves  are  check  valves  and 
which  are  float  valves  when  they  respond  to  the  task  statements.  Additionally,  it  may  not 
make  any  difference  in  the  quality  of  the  data  collected  to  know  which  respondents  work  with 
check  as  opposed  to  float  valves;  it  may  be  sufficient  to  know  that  they  remove  and  rqplace 
valves. 


Equipmait-related  tasks  can  be  mapped  semantically,  although  the  quality  of  the 
mapping  is  a  function  of  the  specificity  of  the  generic  descriptors  used.  The  choice  of 
descriptors  not  only  affects  the  ability  of  subject  matter  experts  to  map  WUCs  to  task 
statements  that  incumbents  indicate  they  perform,  but  can  affect  how  accurately  job 
incumbents  respond  to  the  tasks.  Tasks  presenting  the  greatest  mapping  and  communication 
problems  are  those  for  which  the  differences  of  nomaiclature  across  aircraft  is  such  that  a 
generic  descriptor  is  not  easily  generated  from  the  WUC  nomenclatures.  Development  of 
a  technology  of  generating  tasks  from  MDCS  information  which  incorporates  intelligent  or 
computer-assisted  methods  should  be  possible  and  should  improve  both  present  semantic 
mapping  capability  and  the  specificity  of  weapon  system  task  information. 
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The  last  conclusion  is  that  OSM  tasks  are  both  consistent  and  inconsistent  in  the  way 
they  are  expressed.  They  are  consistent  in  that  task  statements  begin  with  verbs,  or  a  verb 
phrase  (the  action  taken),  followed  by  the  object--the  thing  acted  upon.  They  are  inconsistent 
in  the  way  verb  phrases  are  written.  Sometimes  the  initial  word  of  a  task  statement  is  a  verb 
modifier,  as  in  "Operationally  check."  Verb  phrases  are  employed  in  different  forms,  as  in 
"Isolate  malfunctions  in  air  refueling  systems,"  and  "Isolate  electrical  malfunctions  in 
electrical  circuitry. "  While  there  is  no  communication  problem,  the  different  usages  presait 
problems  in  designing  an  intelligent  system  and  in  semantic  mapping. 


Data  Collection  and  Analysis  Policies 

In  terms  of  survey  policies  and  practices,  occupational  survey  Job  Inventories  contain 
all  of  the  tasks  job  incumbents  in  the  specialty  (or  specialties)  and  the  shred  being  surveyed 
may  perform.  There  are  five  primary  reasons  supporting  this  practice: 

o  All  job  incumbents  have  the  same  frame  of  task  reference  when  responding  to 
survey  instruments. 

o  Job  incumbents  of  maintenance  specialties  firequently  perform  maintenance  tasks 
on  more  than  one  aircraft. 

o  Use  of  a  single  inventory  facilitates  hierarchical  grouping  analysis. 

o  Input  programs  for  ASCII  CODAP  cannot  easily  aggregate  and  analyze  multiple 

inventory  data. 

o  Fewer  resources  are  required  for  preparation,  administration,  and  data  analysis 
of  a  single  inventory. 

The  following  paragraphs  provide  further  explanation  of  each  of  these  reasons. 

First,  all  job  incumbents  have  the  same  frame  of  task  reference  when  they  respond  to 
the  same  set  of  tasks  in  making  thek  relative  time  sp«it,  task  difficulty,  training  emphasis, 
or  other  task  factor  ratings.  Having  the  same  frame  of  reference,  especially  when  rating 
task  difficulty  and  training  emphasis,  is  extremely  important  because  of  the  use  of  relative 
Toting  scales.  If  inventories  are  administered  such  that  job  incumbents  do  not  have  the 
opportunity  to  respond  to  all  the  tasks  describing  the  work  of  a  specialty  (that  is,  different 
subsets  of  tasks  for  different  subsets  of  respondents),  the  task  factor  values  would  apply  only 
to  the  tasks  that  a  particular  subset  of  respondents  rated.  These  factor  ratings  could  not  be 
applied  across  a  specialty  without  the  use  of  techniques  for  equating  ratings  across  the 
complete  set  of  tasks  comprising  the  work  of  an  AFS.  The  equating  problem,  however,  is 
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not  insurmountable;  some  set  of  "common"  tasks  could  be  used  across  sq)arate  subsets  and 
serve  as  "benchmark"  tasks  for  equating  purposes. 

Two  kinds  of  tasks  offer  possibilities  to  minimize  the  benchmarking  issue.  First,  as 
indicated  in  a  preceding  discussion,  every  Job  Inventory  contains  both  general  and  generic 
tasks  which  are  necessary  to  describe  Ae  work  of  a  specialty  but  which  are  not  WUC 
nomenclature-specific.  Opportunity  for  re^nse  to  these  tas^  can  be  provided  to  each 
respondent  and  task  factor  rater  in  a  siuvey.  These  tasks  could  be  benchmark  tasks. 
Second,  generic  tasks  that  are  WUC  nomaiclature-specific,  if  they  can  be  successfully 
general^,  would  provide  a  better  source  of  equating  task  factor  ratings  as  well  as  better 
serve  some  of  the  applications  of  OSM  data. 

In  either  case,  success  depends  upon  the  existrace  of  a  standard  taxonomy  of  basic 
aircraft  systems  and  subsystems,  development  and  consistent  application  of  a  "standard"  set 
of  generic  tasks,  and,  if  necessary,  development  of  a  semantic-assisted  approach  to 
consolidating  similar  nomenclature  from  different  aircraft.  A  standardized  taxonomy  is 
requisite  for  organization  of  tasks  across  aircraft  covered  by  an  AFS.  The  "standard" 
generic  tasks  can  be  generated  from  the  Support  General  and  Inspection  Codes,  which  are 
highly  similar  across  aircraft.  The  SAAT  has  the  potential  for  aggregating  nomenclature  for 
similarly-named  equipment  across  aircraft  for  use  in  generating  a  meaningful  generic 
descriptor. 

Second,  job  incumbents  of  maintenance  specialties  frequently  perform  maintenance 
tasks  on  more  than  one  aircraft.  If  tasks  describing  all  aircraft  are  in  the  same  inventory, 
collection  of  data  from  incumbents  working  on  multiple  airoiaft  is  facilitated.  Because  of 
the  difficulty  in  determining  which  combination  of  aircraft  any  givra  incumbent  maintains 
before  Job  Inventories  are  mailed  for  data  collection,  utilization  of  sq>arate  task  lists  for  each 
aircraft  can  inhibit  data  collection.  Changes  in  present  procedures  for  determining  where  Job 
Liventories  should  be  mailed  for  data  collection  could  possibly  provide  for  incumbent 
responses  on  multiple  aircraft  upon  which  they  work.  These  changes  would  involve  either 
use  of  the  Program  Element  Codes  against  which  job  incumbents  are  assigned,  or  a  pre¬ 
survey  to  determine  data  collection  locations  and  incumbent  requirements  prior  to  mailing 
Job  Inventories.  Computer  administration,  especially  a  computer-adaptive  system,  would 
largely  negate  the  data  collection  issue.  The  major  concern  would  then  be  length  of  the  Job 
Inventory,  since  ASCII  CODAP  will  not  accept  more  than  3,000  items;  however,  a  truly 
adaptive  system  would  not  face  the  length  problem. 

Third,  use  of  a  single  inventory  consisting  of  genetic  tasks  facilitates  hierarchical 
grouping  analysis  to  identify  the  different  gaieric  jobs  in  specialties  crossing  multiple 
aircraft.  Job  groups  specific  to  a  given  aircraft  seldom  can  be  readily  identified.  Separate 
inventories,  or  inventories  consisting  of  q)ecific  tasks  by  aircraft,  would  guarantee  job  types 
peculiar  to  each  aircraft.  Different  incumbents  would  respond  to  different  subsets  of  tasks. 
There  is  considerable  merit  to  these  latter  kinds  of  job  types,  especially  for  use  in 
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determining  personnel  requirements  during  the  system  acquisition  process,  because  of  toe 
specificity  of  job  definition.  The  disadvantage  is  associated  with  toe  inability  to  identify 
similarity  of  toe  work  performed  across  toe  job  groups  that  is  important  for  certain  other 
manpower,  personnel,  and  trainmg  uses.  Some  type  of  benchmarking  tasks,  however,  could 
reduce  or  ftiiminate  the  problem  of  hierarchical  grouping  of  incumbent  data  collected  from 
separate  inventories. 

Fourth,  toe  input  programs  to  ASCII  Comprehensive  Occupational  Data  Analysis 
Programs  (ASCII  CODAP)  for  both  Sperry  and  IBM  mainfirame  computer  analysis  are  not 
designed  so  that  data  from  sq)arate  inventories  can  be  easily  aggregated  and  analyzed.  In 
addition,  no  more  than  3,000  tasks  and  background  information  items  can  be  analyzed  by 
ASCn  CODAP.  If  tasks  are  developed  for  all  aircraft,  whether  they  are  administered  in  a 
single  inventory  (by  mail  or  computer)  or  in  separate  inventories,  toe  number  of  tasks  in 
inventories  for  some  specialties  can  be  expected  to  exceed  toe  maximum  that  can  be 
analyzed.  Thus,  it  will  be  imperative  that  toe  method  or  methods  for  generating  a  single  task 
inventory  accommodate  AFS  for  which  a  multi-aircraft  Job  Inventory  wfil  be  too  long  for 
processing. 

Finally,  fewer  resources  are  required  for  toe  administrative  preparation,  mail 
administration,  and  data  reduction  of  a  single  inventory,  ftivolved  are  costs  of  manuscript 
preparation  and  quality  control;  Job  Inventory  printing,  mailing,  tracking  and  accounting,  and 
quality  control  of  responses;  and  optically  scanning  incumbait  responses.  These  costs  would 
simply  be  multiplied  by  toe  number  of  separate  inventories  used.  Computer  administration 
could  be  expected  to  reduce  some  of  toe  additional  costs  of  "separate"  inventories. 

In  terms  of  data  analysis,  toe  reporting  of  job  data  for  considering  classification 
structuring  issues  and  supporting  training  decisions  are  primary  objectives  of  toe  OSM.  Data 
are  required  which  indicate  toe  degree  to  which  job  incumbaits  of  a  spwialty  or  specialties, 
if  they  are  surveyed  in  a  single  inventory,  paform  toe  same  tasks.  This  information  is  one 
piece  of  information  essential  to  restructuring  considerations.  Important  analysis  tools  are 
composite  job  descriptions  for  various  categories  of  toe  respondent  population,  hierarchical 
grouping  analysis  to  identify  job  groups  and  toe  characteristics  (such  as  AFS  or  experience 
level)  of  toe  job  incumbents  comprising  each  group,  and  comparisons  of  task  factor  data. 
Use  of  multiple  inventories  complicates  these  kinds  of  analyses,  for  reasons  already  cited. 


Tmplipatiops  of  Weapon  Svstem-Specific  Tasks  for  Data  Users 

In  toe  over  20  years  that  toe  Air  Force  Occupational  Analysis  Program  has  been 
operational,  survey  data  have  been  utilized  in  numerous  research  projects  and  for  a  wide 
variety  of  personnel  and  training  purposes.  As  a  result,  toe  OSM  approach  aims  at  collecting 
data  that  can  be  analyzed  for  as  many  purposes  as  possible. 
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Although  the  potential  for  using  these  task  and  job  data  in  acquisition  planning  was 
recognized  almost  at  the  inception  of  the  Program,  it  was  not  exploited.  Only  recently  has 
any  real  thrust  to  capitalize  on  the  data  been  initiated.  It  is  this  thrust  that  produced  this 
research  effort  to  map  OSM  and  MDCS  data  and  to  build  more  weapon  system-specific  task 
lists.  This  effort  if  the  methodologies  produced  by  it  are  successful,  could  not  only  cause 
important  changes  in  the  way  the  USAFOMC  presently  develops,  administers,  and  analyzes 
occupational  surveys,  but  could  produce  significant  changes  in  the  characteristics  of  the  data 
collected  and  rq)orted. 

For  these  reasons,  careful  consideration  must  be  given  to  the  implications  of  changes 
in  the  nature  of  the  data  for  various  survey  data  applications  in  the  design  of  new  approaches 
to  task  development  and  administration.  To  illustrate  the  implications,  this  section  will 
briefly  examine  the  kinds  of  tasks  most  useful  for  three  of  the  major  applications  of 
occupational  data:  structuring  specialties,  training,  and  development  of  test  outlines  for 
Specialty  Knowledge  Tests.  These  three  uses  adequately  show  how  the  kinds  of  tasks 
presently  employed  in  the  OSM  are  useful  in  some  applications  and,  as  well,  how  specific 
tasks  would  fac^tate  their  use  in  other  applications. 


Structuring  Specialties 

Historically,  broad,  generic  tasks  satisfied  the  requirements  for  structuring 
maintenance  specMties,  but  in  recent  years  the  basis  of  classification  is  changing.  Review 
of  the  current  Airman  Classification  Chart  and  specialty  descriptions  in  AFR  39-1  shows  that 
there  is  a  mixture  of  the  ways  maintenance  specialties  are  organized.  In  some  specialties  as 
they  are  presently  structured,  generic  tasks  appear  suitable  -  for  example,  the  Fuels  and  the 
Egress  Systems  specialties.  In  these  cases,  simple  functional  commonality  of  the  equipment 
maintained,  and  thus  similarity  of  system  knowledge  and  skills  which  are  driven  by  the 
functional  similarity,  are  the  bases  for  classification.  Generic  task  statements,  as  used  in 
current  OSM,  satisfectorily  answer  classification  questions,  since  commonality  of  task 
performance  is  the  basic  issue. 

There  are  other  specialties,  however,  in  which  classification  is  based  on  other 
considerations.  Management  of  people,  proximity  of  systems  or  work  location,  and  overall 
weapon  system  or  work  environmental  orientation  (as  opposed  to  similarity  of  system 
knowledge  and  skills)  seem  to  be  more  important  classification  factors  than  functional 
similarity  of  systems  or  specific  functional  knowledge  and  skiUs  requisite  for  work  on 
specific  systems.  Knowledge  and  skills  requirements  are  considered  from  a  broad  base,  such 
as  commonality  arising  because  different  systems  all  reside  on  the  same  aircraft.  An 
example  of  a  specialty  based  on  aircraft  identification  or  orientation  is  AFS  455X2,  F-16 
Avionics  System  Specialist,  that  includes  shreds  through  the  5-skill  level  for  work  on  Attack 
Control  Systems;  Instruments  and  Flight  Control  Systems;  and  Communication,  Navigation, 
and  Penetration  Aids  Systems.  The  commonality  of  fundamental  knowledge  among  these 
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shreds  is  most  apparent  in  proximity,  aircraft,  and  basic  electronics  theory.  This  specialty 
is  a  composite  of  parts  of  three  other  previously-existing  specialties  based  on  commonality 
of  systems  functions  rather  than  aircraft  commonality.  Although  the  new  specialty  structure 
is  based  on  aircraft  identification,  the  integrity  of  the  functional  similarity  of  the  three 
previous  specialties  has  been  perpetuated  by  the  three  shreds.  Since  the  new  AFS  provides 
for  q)ecific-aircraft  orientation,  the  only  kind  of  task  statement  of  value  for  this  and  similar 
AFS  are  very  specific  to  the  systems  on  the  designated  aircraft. 

Similarly,  if  in  the  future  other  generic  specialties  are  to  be  decomposed  into  separate 
specialties,  weapon  system-specific  tasks  and  associated  data  can  be  useful.  The  task  verbs 
used  to  describe  maintenance  actions  (i.e.,  troubleshoot,  align,  adjust,  install,  remove, 
replace,  operationally  check,  test)  classify  the  kinds  of  maintenance  action,  and  associated 
aircraft  systems  knowledge,  and  hands-on  maintenance  skills  as  a  part  of  the  basis  for 
structuring.  Another  prerequisite  of  the  structuring  basis  which  still  must  be  determined  is 
identification  of  the  degree  of  similarity  of  the  knowledge  and  skills  required  by  the  specific 
(as  opposed  to  the  generic)  equipment  items  (the  task  statement  verb  objects)  to  perform  the 
maintenance  actions  (the  task  statement  verbs)  required.  Almost  certainly,  more  precise 
judgments  of  similarity  can  be  made  about  the  specific  objects  than  would  be  possible  from 
general  judgments  made  across  the  objects.  It  is,  of  course,  these  specific  knowledges  and 
skills  that  constitute  the  greatest  training  load. 

As  a  general  rule,  it  is  easier  to  aggregate  than  decompose  tasks  and  their  associated 
data.  This  suggests  that  even  for  generic  specialties,  specific  tasks  could  be  more  useful  for 
considering  both  within  and  across-system  restructuring  issues.  Aggregation  of  the  specific 
tasks  and  associated  data  relating  to  functionally  similar  systems  meaningfully  into  generic 
tasks,  for  the  purpose  of  determining  the  conimonalitv  of  performance  of  similar  tasks, 
presents  the  greater  challenge  -  a  challenge  which  can  be  minimized  by  use  of  a  standard 
aircraft  taxonomy.  The  aggregation  problem  is  caused  by  the  variations  of  terminology 
employed  across  aircraft  to  describe  similar  systems,  and  is  the  same  kind  of  aggregation 
problem  that  inventory  developers  encounter  in  developing  generic  task  descriptors.  It 
should  be  noted  that  OSM  data  only  addresses  commoiuility  of  task  performance;  it  does  not 
provide  information  about  the  similarity  of  knowledge  or  skills  required  for  performance  of 
tasks  that  are  not  coperformed. 

In  conclusion,  both  generic  and  weapon  system-specific  tasks  would  appear  to  be 
desirable  depending  upon  the  classification  structuring  issue.  Any  requirement  could  be 
satisfied  if  both  kinds  of  data  were  available. 


Training 

There  are  applications  for  both  generic  and  weapon  system-specific  tasks  in  training 
decision  making  and  development.  Training  decision  making  involves  deciding  what  to  train 
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in  initial  training  for  generic  specialties  where  the  AFS  has  responsibility  for  maintenance 
of  certain  systems  across  the  force-wide  inventory  or  subsets  of  aircraft.  Generic  tasks  can 
be  more  useful  in  this  process.  Here  it  is  not  a  question  of  which  aircraft  should  be  the 
target  of  training,  but  which  systems  and  what  tasks  across  aircraft  should  be  trained.  With 
specific  tasks,  the  problem  would  reside  in  determining  how  the  separate  tasks  would  be 
aggregated  across  aircraft  to  reach  these  decisions.  Again,  an  aircraft  taxonomy  for  grouping 
tasks  would  be  especially  useful. 

Specific  tasks  would  certainly  appear  to  be  more  useful  for  the  training  developer 
charged  with  designing  training  relevant  to  a  specific  aircraft  system.  Task  specificity  could 
be  of  great  value  in  field,  on-the-job,  and  advanced  training. 


Specialty  Knowledge  Test  Outlines 

As  a  result  of  a  continuing  stream  of  research  by  the  AFHRL,  methodology  for 
developing  outlines  for  Specialty  Knowledge  Tests  (SKI}  used  in  the  Weighted  Airman 
Promotion  System  is  being  implemented.  SKT  are  intended  to  include  items  testing 
p^sonnel  who  make  up  the  AFS  test  population  across  the  requirements  of  the  specialty. 
One  of  the  more  important  guidelines  for  test  item  development  is  the  accessibility  of  study 
reference  materials  if  they  are  used  to  supplement  Career  Development  Courses  to  all 
personnel  to  be  tested.  As  a  result,  test  items  on  q)edfic  equipment  are  gaieraUy  avoided; 
tasks  that  are  more  generic  in  nature  provide  a  better  test  item  development  source.  In  fact, 
according  to  personnel  involved  in  this  research,  specific  tasks  inhibit  the  capability  to 
produce  outlines  and  in  some  cases,  after-the-fact  development  of  general  task  statements 
from  tasks  that  were  too  q)ecific  has  been  necessary.  It  seems  reasonable  to  assume  that  if 
OSM  shifted  to  the  collection  of  data  from  specific  tasks,  some  method  of  aggregating 
specific  tasks  and  associated  data  to  provide  a  general  base  for  testing  is  important. 


Weapon  Svstem-Specific  Task  List  Requirements 

In  Table  4,  the  present  organization  of  weapon-system  specific  task  inventory 
requirements  by  AFS  is  listed.  The  list  does  not  include  specialties  maintaining 
communications-electronic  equipment  (AFS  30xxx),  training  equipment  (AFS  34xxx),  visual 
equipment  (AFS  40xxx),  and  missiles  (AFS  41xxx).  These  kinds  of  equipment  are  normally 
maintained  by  a  single  AFS  responsible  for  all  of  the  systems,  thus  presenting  a  much 
simpler  task  description  process  than  is  required  for  aircraft  maintenance  AFSs  (see  Data 
Sources  for  Task  List  Development,  below).  The  AFS  are  categorized  as  maintenance 
AFSs  for  which  WUC-based  specific  tasks  are  inappropriate;  AFSs  where  the  primary  WUCs 
are  Support  Gaieral  and  Inspection  Codes;  AFSs  which  cross  all  on-aircraft  systems;  AFSs 
with  shreds  that  may  facilitate  inventory  development;  AFSs  with  a  limit^  number  of 
aircraft  or  equipment;  and  off-aircraft  equipment  AFSs  which  require  access  to  separate 
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WUC  manuals.  How  the  AFS  are  distributed  is  important  for  assessing  the  development  and 
data  collection  strategies  for  weapon  system-specific  task  lists,  particularly  from  the 
perspective  of  present  OSM  methodology  in  which  task  list  length  is  a  crucial  issue  when  a 
single  task  list  is  employed  to  describe  hll  of  the  work  of  an  AFS. 

Review  of  Table  4  suggests  several  conclusions  relevant  to  developing  WUC-spedfic 

tasks. 

1 .  There  are  six  AFS  for  which  there  is  little  or  no  need  for  WUC-based  task  lists. 
These  AFS  would  appear  to  be  more  appropriately  described  by  methods, 
materials,  and  processes  generic  task  statements. 

2.  Job  incumbents  (crew  chiefs)  of  three  AFS  with  large  numbers  of  aircraft-based 
shreds  normally  report  work  performed  under  Support  General  codes  (which 
include  Inspection  codes).  The  probability  of  identifying  WUC-based  tasks 
identifying  specific  equipment  is  very  low.  Use  of  the  Support  General  codes, 
which  are  prescribed  by  MIL-M-38769C  (USAF),  Manuals,  Technical:  Work 
Unit  Code,  to  develop  basic  task  lists  for  these  specialties  as  well  as  for  use  in 
job  inventories  for  other  AFS  whose  incumbents  perform  aircnaft-general  tasks, 
is  warranted. 

3.  Development  of  inventories  for  six  AFS  which  cross  all  aircraft  may  be 
anticipated  to  be  the  most  difficult  for  generating  WUC-based  task  lists,  because 
there  are  too  many  aircraft  to  produce  WUC-based  tasks  in  a  single  inventory. 
Some  other  approach  to  inventory  development  and  data  collection  will  be 
imperative  to  accommodate  these  AFS. 

4.  Six  AFS  have  shreds  that  may  facilitate  task  list  developmoit,  at  least  in  the 
sense  of  covering  all  of  the  WUCs  and  ATCs  in  a  single  inventory.  Shreds, 
however,  are  consistent  only  through  the  5-skill  level;  shreds  disappear  or 
change  at  higher  skill  levels.  The  merging  of  shreds  at  the  7-skill  level  may 
mean  that  development  of  inventories  for  these  AFSs  may  present  the  same 
problem  as  AFSs  which  cross  all  aircraft. 

5.  Nine  AFS  cover  a  limited  number  of  AFS  or  equipment.  These  AFS  potentially 
may  be  easily  described  in  a  single  inventory. 

6.  There  are  six  AFSs  for  which  the  WUCs  reported  by  incumbents  are  found  in 
WUC  manuals  separate  from  the  aircraft  WUCs.  Access  to  these  separate  WUC 
manuals  should  be  easily  obtained.  These  AFS  may  be  expected  to  be  covered 
in  a  single  inventory. 
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Table  4.  Weapon  System-Specific  Task  fiiventory  Requirements 
By  Air  Force  Specialty  Code 

Afainfpnanpe  AFS  for  Which  Work  Unit  Code-based  Specific  Tasks  Inappropriate 


427X0,  kfechinist 

427X1,  Corrosion  Control  Specialist 
427X4,  Metals  Processing  Specialist 
427X5,  Airframe  R^air  Specialist 
458X1,  Nondestructive  Inspection  Specialist 
458X3,  Fabrication  and  Parachute  Specialist 

AFS  Where  Primary  Work  Unit  Code  is  Support  General  nr  Tnsncction 


452X4,  TAC  Aircraft  Maintenance  Specialist* 

A,  F-15 

B,  F-16 

C,  F/FB-111 

D,  F-4 

E,  A-10 

F,  A-7 


G,  F-5 

H,  OV-10 

J,  T-38 

K,  T-37,  OA-37 

L,  T-33 

Z,  All  others 


♦The  A,  B,  and  C  shreds  carry  through  the  7-skill  level;  the  remaining  shreds  are  combined 
to  form  the  M  shred  at  the  7-skill  level. 


457X0,  Strategic  Aircraft  Maintenance  Specialist 

A,  B-1 

B,  B-52 

C,  C-18,  all  C-135,  E-3,  E-4,  VC-25,  VC-137 

D,  KC-10 

E,  SR-71,  TR-1,  U-2 
457X2,  Airlift  Maintenance  Specialist** 

A,  C-23,  C-130 

B,  C-5 

C,  C-9,  C-20,  C-22,  C-140,  C-141,  T-39,  T-43 

**The  B  and  C  shreds  carry  through  the  5-skill  level;  they  are  combined  at  the  7-skill  level 
into  a  D  shred. 


AFS  Crossing  All  Aircraft— On-Aircraft  Systems 

423X0,  Electrical  Systems  Specialist 
423X1,  Environmental  Systems  Mechanic 
454X2,  Egress  Systems  Mechanic 
454X3,  Fuel  Systems  Mechanic 
454X4,  Pneudraulic  Systems  Specialist 

A,  MAC  or  SAC  Pneudraulic,  Hydraulic 
457X1,  Helicopter  Maintenance  Mechanic 
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Table  4.  (Continued) 


AFS  with  Sbrads  that  Mav  Facilitate  Inventory  Development  (Some  WUCs  Will  Be  In 
WTTC  Manual  Separate  from  Aircraft  WUCs) 

455X1,  Guidance  and  Control  System  Specialist* 

A,  MAC 

B,  SAC 

C,  TAC 

455X2,  Communications  and  Navigation  Systems  Specialist* 

A,  MAC 

B,  SAC 

C,  TAC 

455X3,  Weapons  Control  SpeciaUst* 

A,  F-4E/G.  AC-130.  F-5E 

B,  A-7D/K 

C,  F-4D 

456X1,  Electronic  Warfare  Systems  Specialist* 

A,  Strategic 

B,  Tactical 

456X2,  Defensive  Fire  Control  Specialist* 

A,  AN/ASG-15 

B,  AN/ASG-21/AN/ASG-33 
462X0,  Aircraft  Armament  System  Specialist 

C,  A-10  J,  FB-111 

D,  F-4  K,  B-52G/H 

E,  F-15  L,  B-IB 

F,  F-16  Z,  All  others 

H,  F-111 

*Shredded  through  the  7-sldll  level;  all  others  through  5-skill  level 
AFS  with  T  imifpiH  Number  of  Aircraft  or  Equipment 

451X4,  F-15  Avionics  Test  Station  and  Computer  Specialist* 

A,  Auto  Test  Sets 

B,  Manual  and  Electronic  Warfare  Test  Station  Console 
451X5,  F-16,  A-10  Avionics  Test  Station  and  Computer  Specialist 
451X6,  F/FB-111  Avionics  Test  Station  and  Computer  Specialist* 

A,  Auto  Equipment 

B,  Manual  and  Electronic  Warfare  Equipment 
452X1,  F-15  Avionics  System  Specialist* 

A,  Attack  Control  Systems 

B,  Instruments  and  Flight  Control  Systems 

C,  Communication,  Navigation,  and  Penetration  Aids  Systems 
452X2,  F-16  Avionics  System  Specialist* 

A,  Attack  Control  Systems 

B,  Instruments  and  Flight  Control  Systems 

C,  Communication,  Navigation,  and  Penetration  Aids  Systems 
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Table  4.  (Concluded) 


AFS  with  limited  number  of  aircraft  or  equipment 

452X3,  F/FB-111  Avionics  System  SpedaUst* 

A,  Attack  Control  Systems 

B,  Instruments  and  Flight  Control  Systems 

C,  Communication,  Navigation,  and  Penetration  Aids  Systems 
455X4,  Airborne  Warning  and  Control  Radar  Specialist 

456X0,  Bomb-Navigation  Systems  Specialist 
457X3,  B-IB  Avionics  System  Specialist* 

A,  Off,  Avionics  Systems,  CITS,  and  Doppler  Radar  Systems 

B,  Instrument  and  Flight  Control  Systems 

C,  Communication,  Navigation,  and  Penetration  Aids  Systems 
♦Shredded  through  5-skill  level 

Off-Aircraft  Equipment  AFS  rRequire  Separate  WUC  INfenuals) 

404X0,  Visual  Information  Equipment  Maintenance  Specialist 
454X0,  Aerospace  Propulsion  Spedalist* 

A,  Jet  Engines 

B,  Turboprop  and  Turboshaft 
454X1,  Aerospace  Ground  Equipment 

455X0,  Photographic  and  Sensors  Maintenance  Specialist* 

A,  TAC/RECON  Hectronic  Sensors 

B,  Recon/Electro-optical  Sensors 
455X5,  Avionics  Support  Equipmait  Specialist* 

A,  F/RF-4  {eci;oar 

B,  A-7/C-5  Avionics 

455X6,  Airborne  Command  Post  Equipment  Specialist 
♦Shredded  through  7-skill  level 


VI.  DATA  SOURCES  FOR  DEVELOPING  TASK  LISTS 


Under  the  Air  Force  Maintenance  Data  Collection  System  each  maintenance  action 
performed  on  Air  Force  equipment  must  be  fully  and  accurately  documented  [MIL-M- 
38769C  (USAF)].  Standard  codes  necessary  for  automatic  data  processing  (such  as  the 
WUC  and  ATC  referred  to  throughout  this  report)  have  been  developed  for  recording  a 
variety  of  data  pertaining  to  the  kind  of  maintenance  performed  for  each  kind  of  equipment. 
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This  section  will  describe  the  types  of  codes  available,  the  basis  of  their  organization,  and 
the  implications  for  developing  WUC-based  tasks. 


Work  Unit  Code  Coverage  and  Organization 

MIL-M-38769C  (USAF)  prescribes  the  requirements  for  the  development  and 
preparation  of  WUC  manuals  for  Air  Force  equipment.  Table  5  displays  the  categories  of 
equipment  for  which  maintenance  data  are  to  be  collected  and  reported.  The  first  two 
characters  of  the  WUC  (the  system  level)  for  a  given  category  of  equipment  maintained  are 
invariant  and  are  the  basis  for  organizing  the  codes  in  the  manual  for  any  given  equipment 
category.  Any  deviation  must  be  pre-approved.  The  table  suggests  that  every  classification 
of  equipment  for  which  there  is  a  maintenance  AFS  is  included  in  MIL-M-38769C  (USAF) 
coverage. 

For  non-aircraft  equipment,  WUC  for  an  equipment  category  is  not  only  invariant  but 
all  maintenance  is  usually  performed  by  a  single  AFS.  For  example,  for  a  missile 
maintenance  AFS,  each  missile  maintained  by  incumbents  of  that  AFS  will  conform  to  the 
WTJC  specifications.  WUCs  designating  airframe  components  will  be  coded  at  the  first  two- 
digit  level  as  11;  wings  and  pinfolds  as  13;  liquid  rockets  as  24,  solid  rockets  as  25,  and  so 
on.  While  the  invariance  of  the  aircraft  WUCs  is  absolute,  the  difference  between  airoraft 
maintenance  and  equipment  maintenance  specialties  is  that  numerous  AFS  maintain  aircraft 
but,  in  most  cases,  a  single  specialty  maintains  all  of  the  systems  on  the  other  equipment. 
Identifying  the  WTJC  for  construction  of  WUC-spedfic  tasks  to  describe  the  non-aircraft 
systems  is  simplified. 

For  aircraft,  the  problem  is  much  more  complex,  because  different  AFS  have  different 
system  responsibilities,  and  each  aircraft  WUC,  except  for  the  standard  two-digit  aircraft 
system  designation,  employs  different  codes  for  subsystems  and  components  even  though 
these  subsystems  and  components  are  similar  in  function  to  those  on  other  aircraft.  There 
is.  however,  a  means  of  reconciling  these  differaices. 


Toward  A  Standard  Aircraft  Taxonomy 

Notice  2,  Appendix  A,  DoD-STD-863B,  Prg)aration  of  Wiring  Data  and  System 
Schematic  Diagrams,  17  March  1981,  provides  a  basis  for  cross  referencing  the  variant 
subsystem  and  component  WTJCs.  DoD-STD-863B  provides  a  text  development  key  and 
a  system/subsystem/subject  numbering  (S/S/SN)  system  that  is  standard  and  applied  to  the 
development  of  all  aircraft.  The  system  codes  in  this  DoD  Standard  replicate  in  a  large  part 
the  Air  Transport  Association  of  America  (ATA)  Specification  100,  Specification  for 
Manufacturer’  Technical  Data,  originally  issued  on  June  1,  1956,  and  reissued  January  15, 
1981.  Variations  between  the  ATA  and  DoD  specification  are  limited  to  the  ATA 
specifications  being  customized  for  DoD  use  in  a  few  places— tike 
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Table  5.  Air  Force  Equipment  Work  Unit  Categories 


Aircraft  Systems  Codes  (e.g.,  Airframe,  Hydraulics,  Instruments,  HF  Communications, 
Vlff  Communications,  Radar  Navigation) 

Air  Launched  Missile  Systems  Codes  (e.g.,  Airframe,  Liquid  Rocket,  Wariiead,  Flight 
Controls) 

Support  Equipment  Systems  Codes  (e.g.,  Servicing  equipment.  Handling  Equipment, 
Guidance  and  Instrumentation,  Mission  Simulators) 

Ground  Launched  Missile  or  Spacecraft  Systems  Codes  (e.g.,  Airframe-Booster 

Structure,  Orbital  Craft,  Electrical  Distribution,  O^ital  Attitude  Maneuvering,  Space 
Ferry  and  Manned  Re-entry  Structure) 

Support  and  Real  Property  Installed  Equipment  Codes  (e.g..  Launcher  and  Launch 

Facility,  Guidance,  Tracking  Network  and  Instrumentation,  Propellant  Loading  and 
storage) 

Munitions  Systems  Codes  (e.g..  Ammunition,  Bombs) 

Communication  Equipm^t  Systems  Codes: 

Standard  Radar  System  Codes  (e.g.,  Antenna,  Transmitter,  Receiver) 

Standard  Computer  System  Codes  (e.g.,  Central  Processing  System,  Input- 
Oulput  System,  Literface  Systems 

Support  General  Codes  (excq)t  for  Communications  Equipment)  (e.g..  Ground  Handling, 
Servicing,  Inspections) 

Support  General  Codes  for  Communications  Electronic  Equipment  (e.g..  Ground 
Handling,  Servicing) 

Aircraft  Support  General  Codes  (e.g..  Look  Phase,  Isochronal,  Special  Inspections) 

Air  Launched  Missile  Support  General  Codes  (e.g..  Look  Phase,  Special  Inspections) 

Air  Launched  Missile  Support  Equipment  Support  General  Codes  (e.g.,  Look  Phase, 
Special  Inspections) 
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Table  5  (Concluded) 

Ground  Launched  Missile  Support  General  Codes  (e.g.,  Look  Phase,  Special  Ihspectioos) 

Non-Nuclear  Weapons  Support  General  Codes  (e.g.,  Look  Phase,  Special  Inspections) 

Peculiar  Munitions  Support  Equipment  Support  Gairaal  Codes  (e.g..  Look  Phase, 

Special  Inq)ections) 

Communications  Equipment  Support  General  Codes  (e.g..  Look  Phase,  Special 
Inspections 

Support  Equipment  Support  General  Codes  (e.g. ,  Look  Phase,  Special  Inspections) 
Training  Equipment  Support  General  Codes 


adding  system  codes  for  systems  on  military  aircraft  not  found  on  civilian  aircraft  (e.g., 
surveillance).  The  DoD  Standard  was  initially  issued  in  1977  and  is  presently  undergoing 
revision. 

DoD-STD-863B  specifies  system  and  subsystem  codes  which  provide  the  basis  for 
identifying  the  WUC  for  a  particular  aircraft.  Associated  with  each  subsystem  code  is  a 
narrative  description  of  sub-subsystems  that  are  to  be  referenced  to  the  subsystem  code.  For 
example,  the  following  narrative  description  is  extracted  for  system  code  29-10,  Main 
Hydraulic  Power,  from  Appendix  A: 

That  portion  of  the  system  which  is  used  to  store  and  deliver 
hydraulic  fluid  to  using  systems.  Includes  items  such  as  tanks, 
accumulators,  valves,  pumps,  levers,  switches,  cables,  plumbing, 
wiring,  external  connectors,  etc.  Does  not  include  the  supply 
values  to  the  using  systems. 

Sub-subsystem  codes  are  selected  by  the  manufacturer  of  an  aircraft.  These  codes,  therefore, 
may  vary  from  aircraft  to  aircraft  and  from  manufacturer  to  manufacturer.  The  only 
restriction  is  that  the  sub-subsystem  codes  and  their  titles  must  conform  to  the  list  of  items 
shown  in  the  narrative  description  of  the  components  of  the  subsystem. 

Examples  of  system  codes  (there  are  a  total  of  99)  and  an  example  of  an  associated 
set  of  subsystem  codes  are  shown  in  Table  6.  The  DoD  Standard  system  Codes  are  not 
identical  to  system  codes  in  WUC  manuals,  but  there  are  one  or  more  easily-accessible  tables 
in  WUC  manuals  that  provide  for  cross-referencing  WUCs  to  the  DoD-STD-863B  system 
and  subsystem  codes. 
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Table  6.  Examples  of  DoD-STD-863B  System  Codes 


Svstem 

Title 

Svstem 

Title 

21 

Air  Conditioning 

39 

Electrical/Electronic  Panels 

22 

Auto  Flight 

&  Multipurpose-Components 

23 

Communications  Crew 

43 

Communications  Staff 

24 

Electrical  Power 

65 

Rotors 

29 

Hydraulic  Power 

99 

Surveillance 

Examples  of  DoD-STD-863B  System-Subsystem  Codes 

Sub- 

Svstem 

Title 

Svstem 

Title 

29 

Hydraulic  Power 

-00 

General 

-10 

Main 

-20 

Auxiliary 

-30 

bidicating 

99 

Surveillance 

-10 

General 

-20 

Data  Display 

-30 

Recording 

-40 

Identification 

-50 

Infra-Red  Sensors 

-60 

Laser  Sensors 

-70 

Surveillance  Radar 

-80 

Magnetic  Sensors 

-90 

Sonal  Sensors 

Inspection  of  WUC  manuals  for  the  F-15,  F-16,  and  B-IB  indicates  that  appendices 
are  included  that  provide  for  cross-referencing  WUCs  to  the  system-subsystem  codes 
prescribed  by  DoD-STD-863B.  The  WUC  manual  for  the  F-15  cross  references  WUCs  to 
the  system-subsystem  specified  in  DoD-STD-863B.  The  F-16  WUC  manual  provides  a  cross 
reference  of  WUCs  to  Reference  Designators,  whose  first  three  characters  (system, 
subsystem)  are  identical  to  the  DoD-STD-863B  system  and  subsystem  codes.  For  example, 
the  first  two  characters  of  the  Reference  Designator  correspond  with  the  DoD-STD-863B 
Functional  System  Number  (e.g.,  29,  Hydraulic  Power);  the  third  character  corresponds  to 
the  DoD-STD-863B  Subsystem  Number  (e.g. ,  29-lx,  Main).  The  B-IB  WUC  provides  both 
Reference  Designators  and  S/S/SN  firom  DoD-STD-863B.  The  preliminary  WUC  maniial 
for  the  C-17  cross  references  S/S/SN  to  WUCs. 
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An  example  of  tracing  WUCs  through  Reference  Designators  to  S/S/SN  for  each  of 
three  aircraft  is  shown  below: 


Aircraft 

WUC 

Reference  Designator 

S/S/SN  fDoD-STD-863BI 

B-IB 

45AAC 

2911FP01 

29-10 

F-15 

45AEC 

291016 

29-10 

F-16 

45AAA 

2911FP1 

29-10 

Reference  to  the  DoD  Standard  system-subsystem  codes  indicates  that  Code  29-1  identifies 
the  Main  Hydraulic  Power  system.  The  WUC  and  Reference  Designator  for  each  of  the 
three  aircraft  are  referenced  to  the  same  S/S/SN:  29-10,  Main  Hydraulic  Power.  Inspection 
of  the  titles  for  each  WUC  will  show  that  the  equipment  is  a  part  of  each  aircraft’s  main 
hydraulic  powCT  subsystem. 

This  referencing  system  provides  the  basis  for  developing,  organizing,  and  aggregating 
MDCS  tasks  across  aircraft  into  OSM  task  inventory  form  for  all  weapon  systems  that  are 
based  on  the  DoD-STD-863B  specifications.  Furthermore  and  most  importantly,  ftig 
referencing  system  will  permit  the  use  of  computer-based  technology  to  accomplish  the  WUC 
linkages  fundamental  to  developing,  organizing,  and  aggregating  WUC-specific  tasks.  At 
this  time,  the  number  of  aircraft  WUC  manuals  available  for  inspection  is  limited  to  the  B- 
IB,  F-16,  F-15,  and  the  C-17.  WUC  manuals  for  other  aircraft  should  be  reviewed  to 
determine  applicability  of  the  DoD  Standard  provisions. 

Assuming  that  WUCs  for  other  aircraft  can  be  referenced  as  described,  MDCS  tasks 
(ATC  plus  WUC  equipment  identification)  for  a  given  AFS  can  be  aggregated  at  the 
subsystem  level.  In  other  words,  all  of  the  actions  taken  on  WUC  equipment  identification 
items  that  are  a  part  of  each  aircraft’s  Main  Hydraulic  Power  System  can  be  aggregated  for 
all  aircraft  under  a  heading  such  as  Maintaining  Hydraulic  Power  Systems  in  an  OSM  Job 
Inventory. 

Or,  if  a  Job  Inventory  for  an  AFS  becomes  too  long,  the  subsystem  codes  and  their 
associated  narrative  descriptions  can  be  the  source  of  development  of  generic  tasks  describing 
maintenance  work  performed  by  incumbents  of  an  AFS  across  an  array  of  aircraft.  There 
are  at  least  three  possible  uses  of  a  generic-based  task  list.  First,  WUCs  for  these  generic 
tasks  can  be  identified  and  precoded  for  each  aircraft  at  the  subsystem  code.  The  precoded 
aircraft  WUC-q)ecific  tasks  would  be  accessible  for  use  in  analysis  after  data  are  collected 
from  job  incumbents  from  the  generic  tasks.  Associated  task  factor  data  would  only  be 
available  for  generic  tasks  if  they  are  the  data  collection  source.  The  generic  data  (mean 
values)  would  be  the  basis  for  estimating  WUC-specific  task  factor  data  by  aircraft.  This 
usage  would  be  analogous  to  the  semantic-assisted  analysis  technology  mapping  of  MDCS 
data  to  OSM  tasks  and  data  in  which  ATCs  and  WUCs  are  mapped  to  OSM  tasks,  but  the 
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associated  task  factor  data  is  based  on  the  generic  tasks,  not  WUC-spedfic  tasks.  The 
difference  is  that  SAAT  mapping  is  after-the-fact  and  is  likely  to  be  less  precise  than  the 
premapping  of  MDCS  data  to  the  generic  tasks. 

Second,  generic  tasks  can  be  a  second  level  set  of  tasks  to  be  used  as  benchmarks  for 
aggregating  task  factor  data  for  incumbents  across  aircraft  for  Job  Inventories  containing 
aircraft  WUC-specific  tasks.  This  aggregation  would  provide  incumbent  data  at  a  generic 
level. 


Third,  generic  tasks  could  be  used  to  describe  work  performed  on  a  subset  of  aircraft 
and  WUC-spedfic  tasks  used  for  another  subset  of  aircraft.  Such  usage  could  be  especially 
applicable  in  Job  Inventories  for  AFSs  having  broad  aircraft  coverage. 

The  important  point  is  that  DoD-STD-863B  provides  an  aircraft  maintenance  structure 
for  computer-generated  task  development  and  data  analysis.  First,  it  is  the  structure  that 
permits  the  mapping  of  WUCs  representing  the  same  functional  equipment  across  aircraft; 
without  it  considerable  effort  would  be  required  to  create  such  a  structure  that  is  absolutely 
essential  for  mapping.  Second,  the  structure  provides  the  potential  structure  for  the  analysis 
of  task  data  from  incumbents.  It  is  the  common  ground  for  aggregating  data  for  different 
aircraft  as  well  as  the  common  structure  for  establishing  benchmarking  tasks  across  aircraft. 
And  finally,  it  provides  the  structure  for  developing  WUC-specific  and  generic  tasks  for  use 
in  redudng  Inventory  length. 


Summary 

This  section  considers  and  amplifies  the  implications  of  information  in  preceding 
sections  for  developing  WUC-based  task  lists  for  OSM  Job  Inventories,  their  administration, 
and  the  analysis  of  data  from  job  incumbents. 

1.  Preliminary  tasks  based  on  WUCs  and  ATCs  can  be  developed  for  use  in  Job 
Inventories  covering  single  and  multiple  aircraft.  These  tasks  will  satisfy  the 
communication  and  clarity  criteria  for  task  description.  Areas  where  precautions 
must  be  exercised  are  in  the  description  of  independent  tasks  and  those 
differentiating  among  workers.  Inventory  development  specialist  intervention 
will  be  required  to  assure  tasks  are  independent  and  differentiate  among 
worlffirs. 

MDCS-based  tasks  can  be  expected  to  reduce  or  standardize  the  various  generic 
tasks  presently  used  to  describe  similar  systems  across  aircraft.  The  general 
tasks  now  found  in  OSM  Job  Inventories  can  become  more  consistent  in 
coverage  through  the  use  of  Support  General  codes. 
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2.  The  present  practice  of  presenting  tasks  in  a  single  inventory  is  well  founded  in 
terms  of  providing  the  same  frame  of  reference  for  all  respondents,  even  if  the 
practice  has  meant  employing  more  generic  task  statements,  consistaicy  with 
present  ASCII  CODAP  processes,  analysis  flexibility,  and  application  of  task 
difficulty  and  training  emphasis  task  factor  information.  Also,  computation  of 
composite  job  descriptions  and  related  data  are  facilitated. 

3.  Use  of  multiple  inventories  in  place  of  a  single  inventory  for  collecting  data  for 
an  APS  would  significantly  increase  the  workload  associated  with  manuscript 
prq)aration  and  quality  control  during  inventory  development  and  mailing, 
follow-up,  data  collection  quality  control,  and  data  reduction. 

4.  The  present  single  inventory  practice  for  data  collection  is  the  most  efficient  way 
of  providing  job  incumbents  who  perform  maintenance  on  several  aircraft  (such 
as  those  assigned  to  consolidated  maintenance  organizations)  to  indicate  the  full 
range  of  their  task  performance. 

In  a  mail  survey,  use  of  sq)arate  inventories  could  be  ejq)ected  to  inhibit  their 
opportunity  to  indicate  the  fuU  range  of  task  performance.  Data  on  the  Uniform  Airman 
Records  (UAR)  file  includes  the  Program  Element  Code  (PEC)  against  which  each  job 
incumbent  is  assigned.  This  information  could  be  included  on  the  mail  distribution  listing 
maHft  up  from  the  UAR  file  to  assure  that  individual  job  incumbents  received  a  Job  Inventory 
relevant  to  their  PEC.  The  problem  of  other  aircraft  maintained,  however,  would  still 
remain  for  mail  administration. 

One  solution  to  the  "other"  aircraft  maintained  by  an  incumbent  idaitified  by  PEC  is 
use  of  a  mail  presurvey.  Incumbents  would  be  surveyed  prior  to  the  mailing  of  the  OSM  Job 
Inventory  to  determine  which  "other"  aircraft  they  maintain.  These  data  could  be  integrated 
in  mailing  lists  obtained  from  the  UAR  file  to  assure  that  each  incumbent  receives  inventories 
for  each  aircraft  maintained.  Such  a  system  increases  OSM  workload,  and  could  become 
administrative,  CODAP  processing,  and  CODAP  analysis  nightmares. 

5.  Computer  administration  methods,  especially  one  modelled  after  computer 
adaptive  testing  methods,  should  eliminate  the  issue  of  opportunity  to  respond 
to  tasks  for  multiple  aircraft  issue.  Computer  administration  would  permit  each 
reqwndent  to  access  all  of  the  tasks  for  each  included  aircraft.  Even  without  an 
adaptive  system,  respondent  access  to  each  of  the  aircraft  maintained  can  be 
provided  by  the  simple  use  of  background  information  items  about  aircraft  (or 
systems)  worked  on,  although  a  computer  adaptive  approach  should  be  much 
more  efficient.  The  technology  (equipment,  communication  media)  exists  for 
computer  administration  following  a  conventional  format,  but  neither  the 
resources  for  accomplishing  it  nor  some  fundamental  issues  bearing  on  electronic 
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data  collection  have  been  investigated.  These  issues  involve  legitimate  research 
questions  that  should  be  resolved  before  computer-based  data  collection  is 
attempted  and  should  be  the  near-term  goal.  Successful  application  of  computer 
data  collection  should  not  minimize  the  pursuit  of  a  longer  term  goal  of 
developing  a  computer  adaptive  methodology.  Achievement  of  the  long  term 
goal  promises  the  potential,  among  others,  of  methods  for  minimizing  the 
number  of  tasks  needed  to  describe  a  specialty  or  specialties  being  surveyed. 

6.  Use  of  weapon  system-specific  tasks  in  at  least  some  AFSs  can  be  expected  to 
produce  task  lists  that  are  too  long  for  ASCII  CODAP  processing.  This 
&cpsctzncy  suggests  that  several  options  for  developing  WUC-specific  tasks, 
including  their  use  along  with  generic  tasks,  will  be  required.  It  is  anticipated 
that  the  length  problem  will  be  encountered  only  for  aircraft  maintenance 
specialties.  The  scope  of  other  non-aircraft  maintenance  specialties  is  more 
limited  and  in  most  cases  a  single  APS  has  responsibility  for  a  given  category 
of  systems.  The  WUC  for  these  non-aircraft  categories  of  systems  provide  the 
structure  for  developing  an  inventory  for  multiple  items  within  a  category. 

7.  Neither  mail  administration  of  separate  inventories  based  on  aircraft 
identification  nor  computer  administration  can  be  employed  to  collect  data  from 
task  lists  that  are  too  long,  at  least  those  exceeding  ASCII  CODAP  processing 
limits.  Further,  even  if  the  lists  are  not  too  long  for  CODAP  processing, 
consideration  of  incumbent  response  load  dictates  using  a  mail  data  collection 
approach  that  minimizes  the  load.  Respondent  workload  can  be  reduced  by  use 
of  tracking  instructions  which  would  guide  respondents  to  inventory  components 
specific  to  their  job  requirements.  Incumbents,  for  example,  working  on  the  F- 
15  would  be  "tracked"  to  the  F-15  WUC-specific  tasks.  Incumbents  who  work 
on  aircraft  for  which  there  are  no  WUC-specific  tasks  would  be  "tracked"  to  the 
listing  of  generic  tasks  describing  other  aircraft. 

8.  A  clear-cut  statement  about  whether  use  of  one  kind  of  task  as  opposed  to 
another  (WUC-specific  v.  generic)  is  preferable  can  not  be  asserted.  Some  users 
can  employ  generic-based  task  ^ta  more  efficiently;  other  users  can  benefit 
from  WUC-specific  task  data. 

9.  MIL-M-38769C(USAF)  provides  information  about  the  WUCs  employed  for  the 
variety  of  equipment  upon  which  maintenance  is  performed  and  recorded  in  the 
MDCS.  For  non-aircr^  equipment,  development  of  WUC-specific  tasks  should 
be  facilitated.  Since  a  single  AFS  normally  has  responsibilityfor  all 
maintenance  of  equipment  items  in  a  given  category  (e.g.,  ground-launched 
missiles,  computers),  the  prescribed  WUC  for  the  category  should  be  sufficient 
for  task  list  development  from  MDCS  data. 
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10.  For  maintenance  of  aircraft,  applying  a  specified  WUC  structure  only  at  the 
aircraft  system  level  complicates  development  of  task  lists  for  aircraft 
maintenance  AFS.  The  structure  in  itself  is  insufficient,  because  of  latitude 
permitted  manufacturers  in  the  designation  of  WUCs  below  the  system  or  first 
two  character  level  of  the  WUC. 

11.  DoD-STD-863B  provides  a  structure  or  aircraft  system-subsystem  taxonomy 
that,  when  used  in  conjunction  with  a  cross  reference  of  Reference  Designators 
to  WUCs  in  the  WUC  manual  for  a  specified  aircraft,  can  be  the  basis  of  a 
computer-generated  system  for  organizing  and  aggregating  WUCs  across  aircraft 
into  OSM  usable  tasks.  The  DoD  Standard  creates  a  system-subsystem 
taxonomy  that  opens  up  a  systematic  approach  to  structuring  aircraft 
maintenance  task  lists  that  will  permit  the  aggregation  of  task  data  collected  from 
job  incumbents. 

In  addition,  the  narrative  descriptions  of  the  content  of  the  sub-subsystem 
specifications  are  sources  for  creating  generic  tasks  useful  in  several  ways.  First,  these 
generic  tasks  could  form  the  task  list  in  Job  Inventories  covering  multiple  aircraft  and 
become  the  basis  for  the  precoding  of  relevant  aircraft  WUCs  for  later  retrieval  and  mapping 
of  aircraft-specific  tasks  after  data  collection. 

Second,  the  generic  tasks  can  be  used  in  conjunction  with  WUC-spedfic  tasks  for  AFS 
with  broad  aircraft  responsibility.  WUC-specific  tasks  can  be  used  to  describe  a  designated 
set  of  aircraft  of  special  interest  and  generic  tasks  used  to  describe  the  remainder  of  the 
aircraft  maintained  by  an  AFS.  Such  usage  assumes  an  organizational  consistency  of  the 
complete  task  list  for  the  AFS,  the  basis  of  which  resides  in  the  DoD  Standard  provisions. 

Third,  and  perhaps  most  important,  generic  tasks  generated  from  the  DoD  Standard 
sub-subsystem  narrative  descriptions  can  serve  as  the  set  of  second  level  tasks  for 
benchmarking  incumbent  responses  across  aircraft.  It  is  unfortunate  that  sub-subsystem 
codes  are  not  specified.  Without  this  specification,  manufecturers  have  the  latitude  to  list 
sub-subsystem  or  component  WUCs  as  their  needs  require;  thus,  an  inconsistency  of  sub¬ 
subsystem  numbering  can  exist  across  the  WUC  for  various  aircraft.  If  the  code 
specifications  were  complete  to  the  fourth  digit,  a  complete  aircraft  taxonomy  would  exist 
for  cross  referencing  across  aircraft. 

A  review  of  WUC  manuals  for  the  F-15,  F-16,  and  B-IB  suggests,  however,  there 
is  reason  to  believe  that  manufacturers  may  be  fairly  consistent  in  identifying  similar  sub¬ 
subsystems  by  the  same  code,  and  to  some  extent,  similar  terminology.  If  this  practice  is 
reasonably  consistent,  which  should  be  revealed  by  examination  of  WUC  manuals  for  other 
aircraft,  development  and  organization  of  generic  tasks  into  a  structure  that  will  facilitate  the 
mapping  of  individual  aircraft  WUCs  to  the  generic  tasks  should  be  possible. 


54 


If  such  a  structure  were  to  be  applied  consistently  during  task  list  development,  tasks 
or  task  data  rq)resenting  a  specific  aircraft  could  be  aggregated  with  tasks  from  other  aircraft 
under  a  generic  task  title;  or,  data  collected  from  generic  tasks  (as  second  level  task 
benchmarks)  could  be  used  to  aggregate  data  across  aircraft  at  the  generic  task  level.  This 
aggregation  of  tasls  and  the  analysis  of  their  associated  percent  performing  and  time  spent 
data  would  be  entirely  proper. 

Further,  even  if  there  is  insufficient  consistency  of  numbering  at  the  sub-subsystem 
level  to  permit  computer-based  linking  procedures  ba^  on  WUCs,  the  Semantic-assisted 
Analysis  Technology  could  provide  with  little  or  no  SME  assistance  the  linkages  of  WUC 
titles  with  generic  tasks  generated  from  the  sub-subsystem  narrative  descriptions.  With  SME 
assistance,  the  system-sub-system  taxonomy,  if  a  narrative  does  not  cover  all  of  the  items 
under  the  sub-system,  could  be  extended  to  a  taxonomy  descriptive  of  all  the  elements  at  the 
sub-sub-system  level.  Then,  since  MIL-M-38769C  (USAF)  provides  that  WUC  equipment 
identification  titles  include  the  noun  which  describes  the  equipment,  it  is  reasonable  to 
hypothesize  that  is  feasible  for  the  SAAT  to  aggregate  WUC  titles  from  different  aircraft  at 
the  noun  and  adjective  level.  Li  fact,  some  aggregation  can  be  expected  to  occur  naturally, 
because  similar  subsystems  will  have  similar  names  across  some  of  the  aircraft  (e.g., 
TACAN  transmitter.  Inflight  Refueling  System).  The  most  difficult  aspect  of  SAAT 
aggregation  can  be  expected  to  be  in  the  inconsistency  of  ordering  of  the  WUC 
nomenclature,  that  is,  the  variable  practices  associated  with  listing  the  noun  first  or  modifiers 
first.  Once  the  WUC  titles  (at  the  noun  and  adjective  level)  are  aggregated,  SAAT  should 
efficiently  link  the  titles  to  the  aircraft-structure  taxonomy.  The  end  result  would  be  generic 
and  weapon  system-specific  tasks  for  data  collection,  analysis,  and  rqiorting.  Both  kinds  of 
users  could  be  satisfied. 

12.  The  implication  of  the  preceding  discussion  is  that  tasks  should  be  developed  at 
two  levels,  specific  and  generic,  for  inclusion  in  a  Job  Inventory.  Li  such 
usage,  special  attention  must  be  paid  to  the  avoidance  of  duplicate  tasks. 

13.  Since  the  Support  General  Codes  (includes  Inspection  Codes)  that  precede  the 
WUC  for  equipment  in  WUC  manuals  are  estabfished  by  MIL-M-38769C 
(USAF),  a  standard  set  of  general  maintenance  tasks  could  be  developed  for 
inclusion  in  each  maintenance  Job  Inventory.  The  Air  Force  Standard  provides 
in  paragraph  30.2.1  that: 
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Support  general  codes  are  for  recording  production 
credit  of  repetitive  tasks  of  a  general  nature  and 
shall  not  be  used  for  recording  malfunctions, 
rq)air,  NRTS  (Note:  not  rq)arable  this  station)  or 
condemnation  actions... All  support  general  work 
must  be  identified  to  the  most  appropriate  code. 
Support  general  code  tasks  not  applicable  to  the 
equipment  being  coded  shall  not  be  included  in  the 
manual. 


The  importance  of  a  standard  set  of  general  maintenance  tasks  is  that  they,  too,  can  become 
standards  for  benchmarking  ratings. 

The  Support  General  Codes  also  are  a  source  for  developing  tasks  for  crew  chief 
(Aircraft  Mechanics)  specialties.  At  the  very  least,  thqr  could  form  the  nucleus  of  a  standard 
set  of  tasks  across  these  specialties. 

14.  Neither  the  number  of  WUC-specific  aircraft  tasks  nor  of  the  generic  tasks  is 
expected  to  be  so  great  as  to  be  inhibiting,  except  when  broad  AFSs  are 
described.  Review  of  the  WUC  codes  as  they  apply  to  a  specific  AFS  for  each 
of  three  aircraft  (F-15,  F-16,  B-IB)  suggests  that  a  relatively  small  number  of 
tasks  will  be  needed  to  describe  the  work  of  the  AFS  on  a  given  airoaft.  In 
addition,  the  DoD  Standard  system  and  subsystem  taxonomy  consists  of  a  small 
number  of  elements  and  the  sub-subsystem  narratives  provide  a  limited  number 
of  categories.  If,  however,  a  task  list  becomes  too  long,  MDCS  data,  such  as 
frequency  of  a  maintoiance  action  for  equipment  items,  could  be  used  as  a 
screening  tool  to  reduce  the  number  of  tasks.  Another  means  of  redudng  the 
number  of  tasks  is  changing  the  level  of  description-from,  for  example,  the  fifth 
character  of  the  WUC  to  the  fourth  character.  Under  current  development 
practices,  changes  in  the  level  of  description  frequently  occur  as  a  way  of 
reducing  task  list  length. 


Conclusions 


With  regard  to  providing  weapon  system-specific  task  data  for  various  manpower, 
personnel  and  training  uses,  the  Semantic-assisted  Analysis  Technology,  with  SME  assistance 
to  verify  and  edit  mapping  of  OSM-MDCS  data,  efficiendy  maps  MDCS  task  level  data  to 
OSM  task  data  from  past  surveys. 
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Weapon  system-specific  tasks  based  on  MDCS  data  can  be  computer-generated  for  use 
in  future  OSM  surveys.  Two  levels  of  task  specification  are  required  for  data  aggregation, 
data  comparability,  and  limitation  of  Job  Inventory  length  when  AFS  are  for  maintenance  of 
two  or  more  aircraft. 

In  the  final  analysis,  methods  for  data  collection  (i.e.,  mail  v.  computer;  presurveys; 
sqparate  inventories)  are  of  lesser  importance  than  the  need  for  an  aircraft  taxonomy  for 
developing  tasks.  Computer  administration,  unless  it  is  an  adaptive  system,  cannot  be 
e;q)ected  to  eliminate  problems  associated  with  aggregation  of  data,  comparability  of  data, 
or  Job  Inventory  length. 

The  crudal  issues  have  to  do  with  development  of  a  methodology  for  linking  WUCs 
from  different  aircraft,  aggregating  data  across  weapon  systems,  making  data  from  aircraft- 
specific  tasks  comparable  across  aircraft,  and  regulating  the  length  of  Job  Inventory  task 
lists. 


The  issues  can  be  addressed  primarily  through  development  of  WUC-spedfic  and 
aircraft-generic  tasks  in  accord  with  an  aircraft  structure  taxonomy.  This  structure 
potentially  resides  in  the  specifications  of  DoD-STD-863B. 

The  following  objectives  should  guide  development  and  use  of  inventories  of  weapon 
system-specific  tasks: 

1 .  Single  inventories  comprised  of  all  of  the  tasks  performed  by  a  specialty  should 
be  used.  Their  use  to  collect  data  should  produce  generic  and  specific  task  data 
to  satisfy  the  different  OSM  data  users. 

2.  Use  DoD-STD-863B  as  the  basic  taxonomy  or  structure  for  developing  weapon 
system-specific  tasks. 

3.  From  DoD-STD-863B  Codes  and  sub-subsystem  narrative  descriptions,  using 
SMEs  if  necessary,  generate  generic  tasks  describing  work  across  aircraft. 

4.  Develop  computer  software  to  map  WUCs  for  individual  aircraft  to  DoD-STD- 
863B  system  and  subsystem  codes  and  sub-subsystem  elements. 

5.  For  AFSs  that  are  for  maintenance  of  a  single  aircraft,  generate  task  lists 
directly  from  MDCS  data. 

6.  For  AFSs  that  are  for  maintenance  of  two  or  more  aircraft,  generate  tasks  at  two 
levels,  WUC-spedfic  and  generic,  for  inclusion  in  Job  Inventories. 


57 


Vn.  RECOMMENDATIONS 


This  section  contains  eight  recommendations  for  weapon  systems-specific  occupational 
analysis  research  based  upon  the  research  lepoited  in  this  document. 

1.  Investigate  use  of  CAMS  as  a  source  of  WUC-AFS  linkage. 

2.  Develop  methods  for  expanding  WUC  nomenclature. 

3.  Investigate  methods  for  improving  selection  of  task-job  descriptions  for  use  in 
SAAT. 

4.  Investigate  use  of  actual  maintenance  tapes  (v.  MDCS  B-4  Master  File)  as 
source  of  WUCs  for  input  into  SAAT. 

5.  Investigate  sources  of  e3q)anded  WUC  equipment  identification. 

6.  Investigate  changes  in  parameters  used  in  SAAT  matching  to  refine  list  of  WUC- 
task  matching  provided  SMEs. 

7.  Initiate  development  of  a  weapon  system-specific  task  list  for  an  APS  covering 
multiple  aircraft.  The  task  list  should  include: 

a.  Tasks  developed  from  the  MDCS  maintenance  data  (v.  B-4  Master  File). 

b.  Tasks  developed  from  the  SuM>ort  General  Codes  provided  in  the  WUC 
manuals  for  each  aircraft. 

c.  Generic  tasks  based  on  the  contents  of  DoD-STD-863B. 

8.  Initiate  investigation  of  development  of  equipment-specific  task  lists  from 
maintenance  data  reported  for  other  equipment  (e.g. ,  communications-electronic, 
missiles,  AGE). 
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X.  LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


AF 

Air  Force 

AFHRL 

Air  Force  Human  Resources  Laboratory 

AFS 

Air  Force  Specialty 

ASCn  CODAP 

ASCn  Comprehensive  Occupational  Data  Analysis  Programs 

ATA 

Air  Transport  Association  of  America 

ATC 

Action  Taken  Code 

CAMS 

Consolidated  Aircraft  Maintenance  System 

LCOM 

Logistic  Composite  Model 

LSAR 

Logistics  Support  Analysis  Record 

MDCS 

Maintenance  Data  Collection  System 

NETS 

Not  Reparable  This  Station 

OSM 

Occupational  Survey  Method 

PEC 

Program  Element  Code 

SAAT 

Semantic-assisted  Analysis  Technology 

SME 

Subject  Matter  Expert 

SKT 

Specialty  Knowledge  Tests 

USAFOMC 

United  States  Occupational  Measurement  Center 

WUC 

Work  Unit  Code 
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